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PRELIMINARY AMENDMENT 

Prior to calculating the claim fees for the above-identified patent application, please 
amend the application as follows to comply with national stage requirements and otherwise 
amend the claims. Please disregard and do not enter the Article 34 Amendment filed during 
international prosecution, which the Applicants do not wish entered at this time. 
In the Specification: 

On page 1, after the title, please insert the following paragraph: 

—Priority Claim 

This is a § 371 U.S. national stage of PCT/US00/25465, filed September 15, 2000, which 
was published in English under PCT Article 21(2), and claims the benefit of U.S. Application 
No. 60/154,601, filed September 17, 1999.-- 
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Please place the following abstract (which is also submitted on a separate page, attached) 
at the end of the specification (i.e., as page 45): 

—SIGNAL COUNTING FOR IN SITU HYBRIDIZATION 

ABSTRACT 

A computer system counts fluorescently tagged nucleic acid probe signals in 
biological specimens by determining a ratio of signals from a test probe to signals of a 
reference probe. Probe signals need not be counted with reference to cells, nuclei, or 
nuclear contours. Gene amplification or deletion can thus be detected by analyzing the 
ratio. Successive image slices are obtained by confocal microscopy, and the images are 
digitized. The digital images are transformed and analyzed to combine contiguous 
fluorescent signal segments in successive optical sections to identify discrete probe 
signals, or spots. Spots overlapping in the axial and transverse dimensions of a three- 
dimensional representation of the biological specimens can be distinguished. A graphical 
user interface presents various features for consideration by a user, who can provide 
guidance to a computer system counting the spots. Various features directed to 
identifying spot clusters and auto fluorescent material can increase accuracy of spot 
counting.— 

In the Claims: 

Please cancel claims 41-57 without prejudice. 
Please amend claims 1, 15, 24 as follows: 

1 . (Amended) A computer-implemented method for counting nucleic acid probe 
signals in a region of interest in a biological specimen, the method comprising: 

in a computer system, automatically counting a number of test signals from a test probe; 

in the computer system, automatically counting a number of reference signals from a 
reference probe; and 
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in the computer system, determining a ratio of the automatically-counted test signals 
from the test probe to the automatically-counted reference signals from the reference probe, 
wherein the region of interest comprises multiple cells. 

2. The method of claim 1, wherein the reference probe is a polynucleotide that 
hybridizes to a centromere, and the number of reference signals from the reference probe 
approximates a nucleus count in the biological specimen. 

3. The method of claim 1, wherein the reference probe recognizes a target on a same 
chromosome as the test probe. 

4. The method of claim 1, wherein the test probe is a polynucleotide that hybridizes 
to a target sequence in a gene, and the reference probe is a polynucleotide that hybridizes to a 
reference sequence. 

5. The method of claim 3, wherein the reference probe recognizes a centromere of 
the same chromosome on which the gene of interest is contained. 

6. The method of claim 1, further comprising obtaining successive images of the 
region of interest to distinguish overlapping signals in the biological specimen. 

7. The method of claim 6, wherein the successive images are optical sections of the 
region of interest. 

8. The method of claim 7, wherein the optical sections are at different depths of the 
biological specimen. 

9. The method of claim 8, wherein the successive images are transformed into 
digital representations in which contiguous signal segments in successive optical sections are 
combined into a single signal in a particular optical section in which a strongest signal segment is 
located. 
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10. The method of claim 6, wherein different successive images are obtained for the 
test probe signals and the reference probe signals, and a quantity of test probe signals and 
reference probe signals are determined. 

1 1 . The method of claim 6, wherein successive images are obtained which show 
distinguishable test probe signals and reference probe signals, and a quantity of the test probe 
signals and reference probe signals are determined. 

12. The method of claim 6, wherein the successive images are obtained by confocal 
microscopy. 

13. The method of claim 1, wherein the ratio of signals is determined without 
reference to boundaries of a cell nucleus. 

14. The method of claim 1, wherein the ratio of signals is determined without 
reference to the boundaries of a cell. 

15. (Amended) The method of claim 1 wherein the probe signals are visible 
signals from probes used with in situ hybridization of a biological sample, the method further 
comprising: 

obtaining a plurality of images at different levels of the biological sample; and 
constructing a three-dimensional image indicating discrete signals at different levels of 

the three-dimensional image; 

wherein automatically counting comprises counting computer-identified discrete signals 

out of the discrete signals at different levels of the three-dimensional image. 

16. The method of claim 15, wherein the three-dimensional image is constructed by 
determining a location of a signal segment in the different levels of the biological sample, 
combining overlapping signal segments in contiguous levels into a single spot signal, and 
separating signal segments in non-contiguous levels into different spots. 
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17. The method of claim 16, wherein the location of signal segments is determined by 
the presence of an increase in brightness intensity that indicates an increase of signal as 
compared to a background signal. 

18. The method of claim 17, wherein the probes display fluorescent signals, and the 
increase in brightness intensity is associated with an increase in fluorescence compared to the 
background signal. 

19. The method of claim 15, wherein the signals comprise test signals from a test 
probe and reference signals from a reference probe. 

20. The method of claim 19, wherein the test probe recognizes a gene of interest, and 
the reference probe recognizes a chromosomal locus having an expected quantity in the 
biological specimen. 

21 . The method of claim 20, further comprising determining a ratio between the test 
signals and the reference signals. 

22. The method of claim 21, further comprising determining: 

(a) whether there is an increase in an expected ratio between the test signal and the 
reference signal, indicating an amplification of the gene of interest; or 

(b) whether there is a decrease in the expected ratio between the test signal and the 
reference signal, indicating relative loss of the gene of interest. 

23. The method of claim 19, wherein the test probe is selected from the group 
consisting of probes that recognize genes implicated or suspected in the development or 
progression of a tumor. 

24. (Amended) The method of claim 15, wherein the biological sample is in a 
microarray. 
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25. The method of claim 24, wherein the microarray comprises a tissue microarray. 

26. The method of claim 25, wherein the tissue microarray comprises tissue samples 
of a same tissue type taken from a plurality of donor specimens. 

27. The method of claim 15, wherein the plurality of images consists of between eight 
and thirty two images at different levels of the biological sample. 

28. The method of claim 15, further comprising: 

avoiding counting discrete signals having intensities exceeding a threshold intensity. 

29. The method of claim 15, further comprising: 

avoiding counting discrete signals having a combined intensity and area exceeding a 
threshold value. 

30. The method of claim 15, further comprising: 

avoiding counting discrete signals related to autofluorescent material. 

3 1 . The method of claim 15, further comprising: 

depicting a two-dimensional image representing the three-dimensional image for 
consideration by a user. 

32. The method of claim 31, further comprising: 

emphasizing discrete signals related to autofluorescent material in the two-dimensional 

image. 

33. The method of claim 15, further comprising: 
identifying a set of one or more discrete signals as a cluster; and 

counting the cluster as a number of discrete signals greater than the number of discrete 
signals in the set. 
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34. The method of claim 33 wherein the cluster is counted as a number of discrete 
signals indicated by applying a mapping to the number of discrete signals in the set. 

35. The method of claim 33 wherein the cluster is counted as a number of discrete 
signals indicated by a function calibrated via manual counting of spots in a plurality of images 



signals indicated by a gain factor applied to the number of discrete signals in the set. 

37. The method of claim 15 wherein the plurality of images are a set of images taken 
during a first analysis of a first color channel, and a second set of images are taken of the 
biological sample for a second color channel, the method further comprising: 

avoiding counting discrete signals appearing at a same location in the set of images for 
the first color channel and the set of images in the second color channel. 

38. The method of claim 15 wherein the plurality of images are a set of images taken 
for a test probe, and a second set of images are taken of the biological sample for a reference 
probe, the method further comprising: 

avoiding counting discrete signals appearing at a same location in the set of images for 
the test probe and the set of images for the reference probe. 

39. The method of claim 15 further comprising: 

receiving a directive from a user indicating counting is to be avoided for a specified 
portion of the biological sample; and 

responsive to the directive, avoiding counting discrete signals for the specified portion of 
the biological sample. 



36. 



The method of claim 33 wherein the cluster is counted as a number of discrete 
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60. A computer-generated user interface for presenting results of microscopic 
observation of biological tissue subjected to a FISH experiment, the user interface comprising: 

a scatter plot of sets of image components designated as spot candidates for the FISH 
experiment; 

wherein the scatter plot comprises points indicating a size and intensity of spot 
candidates. 



61 . The computer-generated user interface of claim 60 wherein at least one point is 
operable to receive a user interface activation to navigate to a user interface display of 
information for a spot candidate associated with the point. 

62. The computer-generated user interface of claim 60 wherein at least one point is 
operable to receive a user interface activation to navigate to a user interface display of a three- 
dimensional depiction of a spot candidate associated with the point. 

63. The computer- generated user interface of claim 60 further comprising: 

a display image depicting a view of the tissue subjected to the FISH experiment and a 
depiction of least one candidate spot thereon; 

wherein the depiction of the candidate spot is operable to receive a user interface 
activation to designate the spot as a minimal intensity spot; and 

wherein candidate spots designated as minimal intensity spots are visually emphasized 
when presenting the scatter plot. 



REMARKS 

The specification has been amended herein to insert Applicant's claim of priority, and to 
insert an Abstract as the last page of the specification. The claims have been amended for 
clarification. No new matter has been added. 

The priority claim to U.S. Provisional Application No. 60/154,601 was already set forth 
in the cover sheet that accompanied the PCT patent application (Application 
No. PCT/US00/25465). The priority claim to the PCT is set forth in the cover sheet 
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40. The method of claim 15 further comprising: 

receiving a directive from a user indicating counting is to be performed separately for a 
specified portion of the biological sample; and 

responsive to the directive, separately counting discrete signals for the specified portion 
of the biological sample. 



41-57 (Canceled) 



58. A computer-readable medium comprising computer-executable instructions for 
performing the following: 

within a stack of image slices generated from a plurality of confocal microscopic 
observations of a FISH experiment as a plurality of depths along a z-axis, identifying possible 
fluorescent image components; 

projecting the possible fluorescent image components within the image slices onto a 
projection image; 

discarding insignificant contiguous possible fluorescent image components in the 

slices; 

for each contiguous region in the projection image, grouping regions of possible 
florescent image components associated with the contiguous region in the projection image into 
spot candidates; 

applying a filter to the spot candidates; and 

counting the remaining spot candidates as spots. 

59. The computer-readable medium of claim 58 wherein insignificant contiguous 
possible fluorescent image components are determined by comparing a size of a contiguous 
possible fluorescent image component with a threshold. 
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accompanying the patent application filed herewith and in the copy of the declaration that is 
submitted with the application filed herewith. 

If any matters remain to be discussed prior to examination, the Examiner is invited to 
contact the undersigned at the telephone number listed below. 



One World Trade Center, Suite 1600 
121 S.W. Salmon Street 
Portland, Oregon 97204 
Telephone: (503) 226-7391 
Facsimile: (503) 228-9446 



Respectfully submitted, 



KLARQUIST SPARKMAN CAMPBELL 
LEIGH & WHINSTON, LLP 
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Marked-up Version of Amended Specification and Claims 
Pursuant to 37 C.F.R. §§ 1.121(b)-(c) 

In the Specification: 

On page 1, after the title, the following paragraph has been inserted: 

Priority Claim 

This is a § 371 U.S. national stage of PCT/US00/25465, filed September 15, 2000, which 
was published in English under PCT Article 21(2), and claims the benefit of U.S. Application 
No. 60/154,601, filed September 17, 1999. 

In the Abstract: 

The attached abstract has been added to the end of the specification. 
In the Claims: 

The claims have been amended as follows: 

1 . (Amended) A computer-implemented method for counting nucleic acid probe 
signals in a region of interest in a biological specimen, the method comprising: 

in a computer system, automatically counting a number of test signals from a test 

probe; 

in the computer system, automatically counting a number of reference signals from a 
reference probe; and 

in the computer system, determining a ratio of the automatically- counted test signals 
from the test probe to the automatically- counted reference signals from the reference probe, 
wherein the region of interest comprises multiple cells. 

2. The method of claim 1, wherein the reference probe is a polynucleotide that 
hybridizes to a centromere, and the number of reference signals from the reference probe 
approximates a nucleus count in the biological specimen. 

3. The method of claim 1, wherein the reference probe recognizes a target on a same 
chromosome as the test probe. 



Page 11 of 18 



.1 O O S 8 i? -A Q - O 9i .1. i& O i 

GLM :dsh 03/1 5/02 E-272-99/1 Attorney Reference Number 4239-62295 

PATENT Express Mail No. EV0532 1 2979 



4. The method of claim 1, wherein the test probe is a polynucleotide that hybridizes 
to a target sequence in a gene, and the reference probe is a polynucleotide that hybridizes to a 
reference sequence. 

5. The method of claim 3, wherein the reference probe recognizes a centromere of 
the same chromosome on which the gene of interest is contained. 

6. The method of claim 1, further comprising obtaining successive images of the 
region of interest to distinguish overlapping signals in the biological specimen. 

7. The method of claim 6, wherein the successive images are optical sections of the 
region of interest. 

8. The method of claim 7, wherein the optical sections are at different depths of the 
biological specimen. 

9. The method of claim 8, wherein the successive images are transformed into 
digital representations in which contiguous signal segments in successive optical sections are 
combined into a single signal in a particular optical section in which a strongest signal segment is 
located. 

10. The method of claim 6, wherein different successive images are obtained for the 
test probe signals and the reference probe signals, and a quantity of test probe signals and 
reference probe signals are determined. 

1 1 . The method of claim 6, wherein successive images are obtained which show 
distinguishable test probe signals and reference probe signals, and a quantity of the test probe 
signals and reference probe signals are determined. 
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12. The method of claim 6, wherein the successive images are obtained by confocal 
microscopy. 

13. The method of claim 1, wherein the ratio of signals is determined without 
reference to boundaries of a cell nucleus. 

14. The method of claim 1, wherein the ratio of signals is determined without 
reference to the boundaries of a cell. 

1 5 . (Amended) [A method of counting] The method of claim 1 wherein the 
probe signals are visible signals from probes used with in situ hybridization of a biological 
[samples] sample , the method further comprising: 

obtaining a plurality of images at different levels of the biological sample; and 
constructing a three-dimensional image indicating discrete signals at different levels of 

the three-dimensional image; and 

wherein automatically counting comprises counting computer-identified discrete 

signals out of the discrete signals at different levels of the three-dimensional image. 

16. The method of claim 15, wherein the three-dimensional image is constructed by 
determining a location of a signal segment in the different levels of the biological sample, 
combining overlapping signal segments in contiguous levels into a single spot signal, and 
separating signal segments in non-contiguous levels into different spots. 

17. The method of claim 16, wherein the location of signal segments is determined by 
the presence of an increase in brightness intensity that indicates an increase of signal as 
compared to a background signal. 

18. The method of claim 17, wherein the probes display fluorescent signals, and the 
increase in brightness intensity is associated with an increase in fluorescence compared to the 
background signal. 
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19. The method of claim 15, wherein the signals comprise test signals from a test 
probe and reference signals from a reference probe. 

20. The method of claim 19, wherein the test probe recognizes a gene of interest, and 
the reference probe recognizes a chromosomal locus having an expected quantity in the 
biological specimen. 



21 . The method of claim 20, further comprising determining a ratio between the test 
signals and the reference signals. 

22. The method of claim 2 1 , further comprising determining: 

(a) whether there is an increase in an expected ratio between the test signal and the 
reference signal, indicating an amplification of the gene of interest; or 

(b) whether there is a decrease in the expected ratio between the test signal and the 
reference signal, indicating relative loss of the gene of interest. 

23. The method of claim 19, wherein the test probe is selected from the group 
consisting of probes that recognize genes implicated or suspected in the development or 
progression of a tumor. 

24. (Amended) The method of claim 15, wherein the biological [samples are] 
sample is in a microarray. 

25. The method of claim 24, wherein the microarray comprises a tissue microarray. 

26. The method of claim 25, wherein the tissue microarray comprises tissue samples 
of a same tissue type taken from a plurality of donor specimens. 

27. The method of claim 15, wherein the plurality of images consists of between eight 
and thirty two images at different levels of the biological sample. 
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28. The method of claim 15, further comprising: 

avoiding counting discrete signals having intensities exceeding a threshold intensity. 

29. The method of claim 15, further comprising: 

avoiding counting discrete signals having a combined intensity and area exceeding a 
threshold value. 

30. The method of claim 15, further comprising: 

avoiding counting discrete signals related to autofluorescent material. 

3 1 . The method of claim 15, further comprising: 

depicting a two-dimensional image representing the three-dimensional image for 
consideration by a user. 

32. The method of claim 3 1 , further comprising: 

emphasizing discrete signals related to autofluorescent material in the two-dimensional 

image. 

33. The method of claim 15, further comprising: 
identifying a set of one or more discrete signals as a cluster; and 

counting the cluster as a number of discrete signals greater than the number of discrete 
signals in the set. 

34. The method of claim 33 wherein the cluster is counted as a number of discrete 
signals indicated by applying a mapping to the number of discrete signals in the set. 

35. The method of claim 33 wherein the cluster is counted as a number of discrete 
signals indicated by a function calibrated via manual counting of spots in a plurality of images. 

36. The method of claim 33 wherein the cluster is counted as a number of discrete 
signals indicated by a gain factor applied to the number of discrete signals in the set. 
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37. The method of claim 15 wherein the plurality of images are a set of images taken 
during a first analysis of a first color channel, and a second set of images are taken of the 
biological sample for a second color channel, the method further comprising: 

avoiding counting discrete signals appearing at a same location in the set of images for 
the first color channel and the set of images in the second color channel. 

38. The method of claim 15 wherein the plurality of images are a set of images taken 
for a test probe, and a second set of images are taken of the biological sample for a reference 
probe, the method further comprising: 

avoiding counting discrete signals appearing at a same location in the set of images for 
the test probe and the set of images for the reference probe. 



39. The method of claim 15 further comprising: 

receiving a directive from a user indicating counting is to be avoided for a specified 
portion of the biological sample; and 

responsive to the directive, avoiding counting discrete signals for the specified portion of 
the biological sample. 

40. The method of claim 1 5 further comprising: 

receiving a directive from a user indicating counting is to be performed separately for a 
specified portion of the biological sample; and 

responsive to the directive, separately counting discrete signals for the specified portion 
of the biological sample. 



41-57 (Canceled) 
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58. A computer-readable medium comprising computer-executable instructions for 
performing the following: 

within a stack of image slices generated from a plurality of confocal microscopic 
observations of a FISH experiment as a plurality of depths along a z-axis, identifying possible 
fluorescent image components; 

projecting the possible fluorescent image components within the image slices onto a 
projection image; 

discarding insignificant contiguous possible fluorescent image components in the 

slices; 

for each contiguous region in the projection image, grouping regions of possible 
florescent image components associated with the contiguous region in the projection image into 
spot candidates; 

applying a filter to the spot candidates; and 

counting the remaining spot candidates as spots. 

59. The computer-readable medium of claim 58 wherein insignificant contiguous 
possible fluorescent image components are determined by comparing a size of a contiguous 
possible fluorescent image component with a threshold. 

60. A computer-generated user interface for presenting results of microscopic 
observation of biological tissue subjected to a FISH experiment, the user interface comprising: 

a scatter plot of sets of image components designated as spot candidates for the FISH 
experiment; 

wherein the scatter plot comprises points indicating a size and intensity of spot 
candidates. 

61 . The computer- generated user interface of claim 60 wherein at least one point is 
operable to receive a user interface activation to navigate to a user interface display of 
information for a spot candidate associated with the point. 
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62. The computer-generated user interface of claim 60 wherein at least one point is 
operable to receive a user interface activation to navigate to a user interface display of a three- 
dimensional depiction of a spot candidate associated with the point. 



63. The computer-generated user interface of claim 60 further comprising: 

a display image depicting a view of the tissue subjected to the FISH experiment and a 
depiction of least one candidate spot thereon; 

wherein the depiction of the candidate spot is operable to receive a user interface 
activation to designate the spot as a minimal intensity spot; and 

wherein candidate spots designated as minimal intensity spots are visually emphasized 
when presenting the scatter plot. 
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5 SIGNAL COUNTING FOR IN SITU HYBRIDIZATION 

Abstract 

A computer system counts fluorescently tagged nucleic acid probe signals 
10 in biological specimens by determining a ratio of signals from a test probe to 

signals of a reference probe. Probe signals need not be counted with reference to 
cells, nuclei, or nuclear contours. Gene amplification or deletion can thus be 
detected by analyzing the ratio. Successive image slices are obtained by confocal 
microscopy, and the images are digitized. The digital images are transformed and 
15 analyzed to combine contiguous fluorescent signal segments in successive optical 

sections to identify discrete probe signals, or spots. Spots overlapping in the axial 
and transverse dimensions of a three-dimensional representation of the biological 
specimens can be distinguished. A graphical user interface presents various 
features for consideration by a user, who can provide guidance to a computer 
20 system counting the spots. Various features directed to identifying spot clusters 

and autofluorescent material can increase accuracy of spot counting. 
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SIGNAL COUNTING FOR IN SITU HYBRIDIZATION 

FIELD OF THE INVENTION 

This invention relates to methods of counting probe signals in biological 
specimens, such as probe signals produced by in situ hybridization in cells or tissue 
sections. 



Recent advances in molecular medicine have provided a greater opportunity 
to understand the genetic basis of disease, as well as the cellular mechanisms of 
disease, and select appropriate treatments with the greatest likelihood of success. 
Such diagnostic and prognostic cellular changes include the presence of tumor 
specific cell surface antigens (as in melanoma), and genetic abnormalities (such as 
activated oncogenes in tumors). A variety of techniques have evolved to detect the 
presence of these cellular abnormalities, including immunophenotyping with 
monoclonal antibodies, in situ hybridization with probes, and DNA amplification 
using the polymerase chain reaction (PCR). 

One such technique for molecular diagnosis is in situ hybridization (ISH) in 
which labeled hybridizing agents (such as DNA, RNA, or single stranded or double 
stranded DNA probes) are exposed to intact tissue sections. The probes can be 
labeled by direct or indirect means. In direct labeling, the label (a chromophore) is 
integral to the probe. Direct labels include fluorescent dyes such as derivatives of 
rhodamine, fluorescein, and Texas Red, or enzymatic reporters such as horseradish 
peroxidase or alkaline phosphatase. Indirect labeling involves attaching a hapten 
(such as biotin, mercury or dioxygenin) to a probe through a linker. After 
hybridization, the hapten is detected using a labeled antibody or other specific 
binding protein. 

When fluorescent dyes are used as labels, the technique is referred to as 
fluorescent in situ hybridization (FISH). Dyes such as fluorescein isothiocyanate 
(FITC) are often used to label a sequence of probe DNA in FISH. The probe 
hybridizes to a defined target nucleotide sequence of DNA in the cell, and the FITC 
fluoresces green when excited by a mercury arc lamp or Argon laser (in the case of 
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laser scanning microscopy), so that the labeled probe can be visually detected when 
the probed tissue is viewed through a microscope. Each chromosome containing the 
target DNA sequence will produce a fluorescent signal (sometimes called a spot) in 
every cell when the specimen is illuminated with suitable excitation. For example, 

5 specimens hybridized with a probe hybridizing to a specific region on chromosome 
21 will produce two fluorescent signals in cells from normal individuals and three 
signals from Down's Syndrome (Trisomy 21) subjects who have an extra 
chromosome number 21 . Alternatively, a fluorescent probe that hybridizes to an X 
chromosome will give one fluorescent signal per cell in males (who have only one X 

10 chromosome) and two fluorescent signals in females (who have two X 
chromosomes). 

FISH is an excellent method for detection of gene copy number alterations in 
cancer and other diseases. FISH is a standard tool for analyzing congenital genetic 
alterations in clinical diagnostics (such as numerical chromosomal alterations, 

15 duplications, inversions and microdeletions). In cancer, characteristic gene 

amplifications or deletions are associated with the development and progression of 
the tumor. FISH analysis of gene amplifications in cancer can have prognostic and 
therapeutic significance, such as the detection of the HER-2 oncogene amplification 
in breast cancer. Detection of this oncogene by FISH was recently approved by the 

20 Food and Drug Administration as a diagnostic tool for human breast cancer. 

However, a limitation on the more widespread use of FISH technology has 
been that counting of the fluorescent spots is extremely tedious, inaccurate, often 
highly subjective, and subject to substantial intra-observer variability. It also 
requires a highly trained technician who can recognize the cells or tissue to be 

25 analyzed and recognize and count tiny fluorescent spots accurately. Finally, at most 
1 00 or 200 cells are typically analyzed per specimen, and in the case of gene 
amplification, much less than that (such as 20 per specimen). This can lead to 
statistical inaccuracy in defining the correct copy number. 

A significant impediment to the accurate counting of fluorescent signals is 

30 that the probes hybridize throughout a three-dimensional nucleus, and the probe 

signals have to be counted from different focal planes for each nucleus. However, 
cells and probe signals can overlap in the two-dimensional view, and overlapping 
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signals are seen as a single signal. Such overlapping results in undercounting of 
signals, which can make it appear that an amplified gene is less amplified, or that 
fewer copies of a normal gene are present. 

Given the tedium and subjectivity of signal counting, efforts have been made 

5 to automate this technique. For example, U.S. Patent No. 5,523,207 discloses a two 
dye FISH method in which probes are labeled with a first dye (such as FITC) and a 
contour of the nucleus is labeled with a second dye (such as propidium iodide or PI). 
These two dyes allow the number of signals per visualized nucleus to be determined. 
However, automated FISH spot counting using such techniques has been limited 

10 because FISH signals in the nuclei are often at different focal planes, resulting in 

interfering out-of-focus light. Moreover, automated detection of nuclear boundaries 
has been very difficult to perform in tissue sections. These factors have contributed 
to the inadequacy of existing algorithms for performing automated FISH. 

It would therefore be helpful to provide a method and device for improving 

15 the accuracy of FISH spot counting. 

SUMMARY OF THE DISCLOSURE 

In one embodiment disclosed herein, fluorescently tagged nucleic acid probe 
signals are counted in a region of interest in a biological specimen by determining a 
20 ratio of signals from a test probe to signals of a reference probe, and the region of 
interest includes multiple cells. This is a contrast to prior approaches, in which 
probe signals have been counted with reference to cells or nuclei, and in which 
automated methods have counted probe signals with respect to stained nuclear 
contours. 

25 In certain embodiments, the reference probe may be a fluorescently labeled 

polynucleotide (such as DNA or RNA) that hybridizes to the region of interest in a 
gene, and the reference probe is a polynucleotide labeled with a different fluorescent 
color, and which hybridizes to a reference target. The test probe may hybridize to a 
gene that is implicated or suspected to be involved in a particular disease, such as 

30 tumor development and progression. The reference probe may, for example, 

recognize a centromere of the same chromosome on which the gene of interest is 
contained. An increased ratio of test probe signals to reference probe signals would 
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then indicate an amplification in gene copy number, while a decrease in that ratio 
would indicate a relative loss of the gene of interest (such as a gene deletion). By 
determining a ratio between the quantities of test and reference signals, the problem 
of measuring changes in gene copy number with respect to a cell or nucleus is 

5 avoided. Typically, a count of centromeres approximates a nucleus or cell count. 

Particularly accurate counting of FISH signals can also be accomplished by 
obtaining successive contiguous images of the region of interest to distinguish 
overlapping signals from the biological specimen. Without distinguishing the 
overlapping signals, they would otherwise obscure one another, and diminish 

10 accuracy of the spot count. In particular embodiments, the successive images are 
slices, such as digital microscopic optical sections from different depths of the 
biological specimen, obtained by confocal microscopy. The successive images are 
transformed by detecting and representing the positional values in each image of 
fluorescent emission signal segments, which make up the probe signal, as an array of 

15 digital values. Signal segments below a certain value (e.g. a threshold number of 
pixels) may be eliminated. The remaining digital signal segments may then be 
analyzed to combine contiguous fluorescent signal segments in successive optical 
sections into a single spot signal, which may be assigned to a particular optical 
section in which a strongest fluorescent signal segment is located, or a group of 

20 optical sections across which the contiguous signal segments have been detected. 
This localization of the fluorescent signal to a particular optical section allows 
overlapping spots to be distinguished, both in the axial and transverse dimensions of 
the three-dimensional representation. 

The system detects the location of fluorescent spot signals in three- 

25 dimensional space by performing a morphological top-hat transform to digital 

images of the different levels to obtain fluorescent intensity spikes that indicate a 
spot signal segment. A threshold level of fluorescence intensity is determined to 
eliminate signal segments that are below a fluorescence intensity that would be 
associated with a valid spot signal. Remaining contiguous spot signal segments are 

30 segmented into a single spot signal, and the single spot signal may be assigned a 
location at a level associated with a greatest fluorescent intensity signal segment. 
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Certain disclosed embodiments also include devices for counting signals 
from in situ hybridization of probes in biological tissue, or determining a three- 
dimensional relationship between the signals, in which the device includes a 
confocal microscope, a digital camera positioned to obtain digital optical section 

5 images of different levels of the biological tissue, and a computer implemented 

system that detects and combines contiguous or adjacent signal segments (which are 
above a threshold) at the different levels, and separates vertically overlying, 
transversely overlapping, or non-contiguous signals from one another. After 
separating such signals from different levels into different spot signals, the computer 

10 implemented system then counts the spot signals, or compares their relative 

locations in three-dimensional space. Two or more different signal types (such as 
two or more distinguishable fluorescent dyes) may be used, one color for the test 
probe signal and a second color for the reference probe signal. The ratio of the 
number of test probe signals to reference probe signals can then be determined, or an 

15 unexpected overlap of the signals (as in a genetic translocation) can be assessed. 

The test probe signals and the reference probe signals may be obtained 
_ separately, for example by successively illuminating the tissue specimen with light 
of different colors that selectively causes the different dyes to fluoresce, by viewing 

the specimen through filters that filter out signals (such as a filter that removes 

~ ... * 

20 colors except the color of interest), or exposing separate contiguous tissue sections 

to the different probes. Alternatively, the test and reference probe signals can be 

obtained simultaneously, using multiple band-pass excitation and/or emission filters. 

Once the test and reference probe spot signals have been counted, a ratio of test 

probe signals to reference probe signals is calculated (without reference to 

25 boundaries of a cell or nucleus) to determine whether there is a genetic alteration, 
such as an alteration in gene copy number. 

Alternatively, in certain disclosed embodiments, the FISH spots are 
accurately counted by obtaining the plurality of digital optical images at different 
levels of a biological sample, and constructing a three-dimensional image showing 

30 discrete fluorescent signal segments at different levels of the three-dimensional 

image. The three-dimensional image is constructed by determining a location of a 
fluorescent signal segment of a particular color in the different levels of the 
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biological sample, combining contiguous signal segments (which are above a pre- 
selected threshold) into a single spot signal, and separating non-contiguous signal 
segments into different spots signals. The location of signal segments in each level 
is determined by the presence of a fluorescent brightness intensity spike that 

5 indicates an increase in image component intensity as compared to a background 
intensity. The locations of signals of different colors can be similarly resolved in 
three-dimensional space, the number of spot signals of each color counted, and a 
ratio of the spot signals determined. This three-dimensional imaging would also 
allow one to study the orientation or location of the spots in the nuclei to make 

10 conclusions about the presence of genetic rearrangements that do not change copy 
number of the signals, such as translocations and inversions. 

Certain features can be implemented to increase the accuracy of the spot 
counting. For example, under certain circumstances, a group of spots may be 
packed so closely together as to form a cluster that is difficult to analyze using a 

15 standard algorithm. Clusters can be automatically identified and counted using a 
cluster calibration feature, or a user can specify that a particular region of an image 
is a cluster. 

Another feature for increasing accuracy filters out false spots by, for 
instance, eliminating spots appearing at a same location in the test and reference 

20 probe images or filtering out image data indicating autofluorescence. In addition, a 
user interface can be presented to allow a user to either select areas of particular 
interest for separate processing or specify areas not to be processed. In this way, the 
invention can benefit from information provided by a human operator. 

In the copy number analysis, the ratio of signals can be a ratio of spot signals 

25 from a test probe that recognizes a gene of interest, and a reference probe that 
recognizes a chromosomal locus having an expected quantity in the biological 
specimen. The method can further include determining whether there is an increase 
in an expected ratio between the test signals and the reference signals, indicating an 
amplification of the gene of interest, or whether there is a decrease in the expected 

30 ratio between the test signals and the reference signals, indicating relative loss of the 
gene of interest. 
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This method is particularly applicable to high throughput techniques for 
performing automated FISH analysis of a large number of tissue, cell, or other 
specimens. The specimens can, for example, be in a tissue or cell microarray that 
includes specimens from the same or different sources, -such as tumors. In such 

5 embodiments, the method includes providing an array of biological samples, 

hybridizing the biological samples with a fluorescent test probe that hybridizes to a 
gene of interest in the biological samples and with a fluorescent reference probe that 
hybridizes to a chromosomal reference locus in the biological samples. Images are 
then obtained by confocal microscopy of contiguous sections at different depths of a 

10 plurality of the biological samples in the array, and fluorescent signal segments from 
the contiguous sections are detected. The contiguous signal segments in different 
sections are combined into a corresponding single spot signal, and the separate spot 
signals are resolved from one another. The distinct spot signals can then be counted. 
In particular embodiments, the tissue array can include an array of many 

15 different tissue specimens, for example at least 50 tissue specimens, but hundreds or 
even thousands of tissue specimens can be included in the microarray. The tissue 
arrays can be constructed by obtaining a plurality of donor specimens, placing each 
donor specimen in an assigned location in a recipient array, and obtaining a plurality 
of copies of the recipient array in a manner that each copy contains a plurality of 

20 donor specimens that maintain their assigned locations. For example, a set of tissue 
samples relating to a same type of tissue from a plurality of donor specimens can be 
included. However, tissue arrays made by any method are suitable for use with the 
method of counting FISH spot signals. 

The foregoing and other objects, features, and advantages of the invention 

25 will become more apparent from the following detailed description of disclosed 
embodiments which proceeds with respect to the accompanying drawings. 

BRIEF DESCRIPTION OF THE FIGURES 
FIGS 1A and IB are schematic side and top views illustrating the problem of 
30 viewing three-dimensional FISH images in two dimensions. 
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FIG. 2A is a schematic view illustrating the prior approach of counting two 
color FISH spot signals with respect to cell nuclei (the contours of the nuclei being 
shown around the spot signals). 

FIG. 2B is a view similar to FIG. 2A, but shows counting two color FISH 
5 spot signals and determining a ratio of the different colored signals in a region of 

interest (such as a field of view FOV) instead of with respect to cell nuclei. FIG. 2C 
is a photomicrograph of prostate tissue, illustrating a region of interest (ROI) within 
the tissue for purposes of calculating a ratio of test to reference probe signals. 

FIG. 3 is a schematic view illustrating two color FISH, in which signal 
10 locations have been determined in three-dimensional space. 

FIGS. 4 A and 4B are flow diagrams illustrating one embodiment of an 
automated system for counting FISH spot signals in accordance with the present 
invention. 

FIG. 5A is a flow diagram illustrating an algorithm for counting the FISH 

15 signals. 

FIG. 5B is a series of optical sections 1-5, and a max image M, of prostate 
tissue subjected to two color FISH with a probe for the androgen receptor (labeled 
red) and a probe for the normal X chromosome (labeled green), in which red signals 
appear weaker than green signals. 

20 FIG. 6A is a composite figure showing a print-out of optical sections of a 

series of eight confocal images (cuts 1-8), and a view of the image that would be 
seen (max-image) when the series of eight overlapping images would be viewed 
from above as a two-dimensional image. 

FIG. 6B is a series of optical sections 1-5, and a max image M, of breast 

25 cancer tissue subjected to two color FISH with a ribosomal S6K probe (labeled red) 
and a probe for the chromosome 1 7 centromere (labeled green), in which red signals 
appear weaker than green signals. The excess of red S6 kinase signals over green 
reference signals indicates amplification of kinase. 

FIG. 7 is a MATLAB graphical user interface (GUI) for displaying optical 

30 sections of a three-dimensional tissue specimen that has been subjected to FISH. 
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FIG. 8 is a histogram of the number of occurrences of the various gray 
levels, or fluorescence intensities, in the max image, which is used to threshold valid 
pixels and eliminate non-informative image components. 

FIG. 9A is a print-out of a MATLAB GUI illustrating processing spot signal 
5 segments from different levels, and combining contiguous signal segments above a 
threshold into a single spot signal (which can be assigned to a particular level or 
levels) for purposes of counting a spot signal. 

FIG. 9B is a schematic view of a vertical side view of optical sections, 
illustrating how image components below a certain threshold are eliminated to 
10 separate otherwise vertically contiguous signal segments, and separate overlying 
signals from one another. 

FIG. 9C is a view similar to FIG. 9B, showing how transversely and 
vertically overlapping signal segments are separated by eliminating signal segments 
below a threshold, and grouping the remaining signal segments into discrete spot 
15 signals. 

FIG. 1 0 is a three-dimensional representation of signal segments in different 
optical sections, resolved into a spot in an assigned optical section, and a combined 
image of the signals projected as spheres into a max image, in which a volume of the 
spheres is proportional to an intensity of the signal. 
20 FIG. 1 1 is a view of a tissue section mounted on a slide, for processing by 

FISH. 

FIG. 1 2 is a view of a tissue microarray mounted on a slide, for processing 
by FISH. 

FIG. 13 is a flowchart showing an overview of an algorithm used to count 
25 FISH spots in a set of digital image slices. 

FIG. 14 is a data flow diagram of an algorithm used to count FISH spots in a 
set of digital image slices. 

FIG. 1 5 A is an image showing FISH spots relating to the X centromere in 
normal prostate tissue, including false spots caused by autofluorescence. 
30 FIG. 1 5B is a graph showing intensities of spot candidates for purposes of 

removing spots exceeding a threshold intensity. 
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FIG. 1 5C is a graph showing intensities combined with area for spot 
candidates for purposes of removing spots exceeding a threshold. 

FIG. 1 5D is the image of 1 5 A modified to emphasize spot candidates 
identified as autofluorescent tissue particles. 

FIG. 16A is a scatter plot of spot candidates for an image relating to a FISH 
experiment conducted for centromere 17. 

FIG. 16B is a scatter plot of spot candidates for an image relating to a FISH 
experiment conducted for the gene HER-2 and identifies spot clusters on the plot. 

FIG. 1 7 is an illustration of a user interface for selecting areas of interest to 
be considered when calculating a spot count for an image. 

FIG. 18 is an illustration of a user interface depicting a three-dimensional 
representation of a spot candidate. 

DETAILED DESCRIPTION OF SEVERAL EXAMPLES 
The present invention includes a method and apparatus for counting FISH 
spots, and particularly an automated, computer implemented method and device that 
helps avoid counting errors introduced by vertically or horizontally overlapping 
spots in two-dimensional projections of three-dimensional tissue sections. One of 
the problems solved by this invention is illustrated in FIG. 1 A, which shows a 
vertical section through a stack of contiguous horizontal confocal layers of a tissue 
section that has been subjected to FISH. Hybridization probe signals are shown as 
spots in this figure, in which it can be seen in a vertical section that the spots are 
discrete and spaced along the z-axis. However, when viewed from above (in an x-y 
plane) as shown at the bottom of FIG. 1 A, the discrete spots overlap and can not 
clearly be counted as separate spots. FIG. IB illustrates a similar problem in which 
even more probe spots appear throughout the z-depth of the tissue section, but 
appear as an indistinguishable blur when viewed in an x-y plane from above, as 
shown at the bottom of FIG. IB. 

Another problem with the prior art is illustrated in FIG. 2A, which shows the 
conventional technique of counting hybridization signals within each cell nucleus. 
This view illustrates two color FISH, in which two probes are labeled with different 
dyes that fluoresce with different colors. The red label (R) may, for example, be 
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attached to a probe that hybridizes to a gene of interest (such as a hormone receptor 
gene that may be amplified in certain tumors). The green label (G), in contrast, may 
be attached to a probe that hybridizes to a known chromosomal locus that is not 
expected to vary in disease states (such as the centromere of a chromosome on 
5 which the gene of interest is found). The red label (R) is illustrated by a gray color 
in the schematic figure, while the green label (G) is represented by a darker color. 

In a particular example, the gene of interest could be recognized by a probe 
for a gene on the X chromosome labeled with spectrum orange (to provide an 
orange-red spot signal), and the reference probe could be labeled with spectrum 

10 green (to provide a green spot signal) for the centromere of the X-chromosome. A 
single green signal (G) would therefore be observed in the nuclei of the schematic 
representation of FIG. 2A from male cells (which have only one X chromosome), 
while two green signals (G) would be seen in female cells. Amplification of the 
gene of interest would be noted in certain cells in the schematic representation of 

15 FIG. 2 A in which there is an increase in the ratio of red signals (R) to green signals 
(G). 

As shown in FIG. 2A, it is conventional to count the number of signals in 
each nucleus of a large number of cells. This approach has been adopted because 
amplification or deletion of a gene occurs in large populations of cells, and 

20 significant changes in copy numbers of genes are often only detected by examining a 
large number of cells (for example, at least 200). Because the amplification has 
been considered to be a nuclear event, a change in the copy number of a gene with 
respect to each nucleus has been counted, both manually and in automated systems. 
Such systems have been difficult to automate, however, because cells and nuclei 

25 overlap (as shown by the overlapping nuclear contours in FIG. 2A), and the nuclear 
contours have been difficult to reliably recognize in automated systems. 

The present invention adopts a different approach which has been found to 
be more accurate, and has the additional advantage of being more accurately 
automated. This approach is shown in FIG. 2B, in which the ratios of probes are 

30 determined without reference to the cells (or the nucleus) in which the probes are 
contained. The nuclear contours highlighted in FIG. 2A are absent in FIG. 2B to 
illustrate this difference. Hence, FIG. 2B shows the FISH spots of FIG. 2A in a 
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region of interest (such as the microscope field of view [FOV] shown in FIG. 2A), 
but without reference to the nuclear contours illustrated in FIG. 2A. In accordance 
with the present invention, the ratio of test probes (R) to reference probes (G) in the 
region of interest is the ratio that is calculated. It has been found that this ratio 
5 provides sufficient information (over a sufficiently large number of cells in a region 
of interest) to be informative about the relative amplification or deletion of a gene of 
interest. 

A region of interest is any arbitrary informative region across which an 
informative ratio can be determined. In some instances, a region of interest is a 

10 microscopic field of view at low magnifications (e.g. 100-200X). An entire 

microscope field of view can then be used for the image capturing and analysis at 
400-1000X magnification (X40-100 objectives) for FISH analysis. The thickness of 
the tissue sections used for FISH analysis is the same as in sections routinely used 
for histopathological analyses, ranging from 4-10 ^ in thickness. 

15 Another example of a region of interest is an area which is selected for a 

specific analysis, as shown in FIG. 2C, which is a cross-sectional photomicrograph 
of tissue from a human prostate, as it would appear in a tissue microarray. In this 
figure, the region of interest (ROI) is the area of malignant cells that is outlined in 
black. A region of interest in this example is any homogenous site of a specimen, 

20 where most, if not all, specimens carry a particular alteration. However, 

contamination with non-altered normal tissue in the region of interest can be 
tolerated, if the copy number alteration in the abnormal cells is substantial. 

In some embodiments, to further improve the ability of the system to 
accurately determine the ratio of test probe signals to reference probe signals, a view 

25 of the probe signals in three-dimensional space is constructed, as shown in FIG. 3. 
This view shows the three-dimensional relationship of the test and reference probe 
signals, which are interspersed among one another in all three dimensions of the 
tissue section which has undergone FISH. This three-dimensional view of the 
section can be further processed by a computer implemented system, as described 

30 below, to automatically count signals of each color, and to obtain a ratio of the 

different colored signals. This approach is particularly useful for high-throughput 
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analysis, for example, of tissue microarrays such as those disclosed in PCT 
publications WO US99/04000 and US99/04001. 

In yet other embodiments, the three-dimensional imaging can be used to 
detect many kinds of genetic rearrangements in cells other than deletions or 

5 amplifications. For example, differently colored probes for the bcr and abl genes 
could be used to detect a fusion of these genes which occurs in a genetic 
translocation associated with chronic myelogenous leukemia (CML). See Tkachuk 
et al., Science 250:559-562, 1990. Such differentially labeled probes will flank 
translocation breakpoints, and produce a fusion after the translocation has occurred, 

10 but will be at separate loci if the translocation has not occurred. However, unless 
proximity or distance between the differentially labeled spots can be determined in 
three-dimensions, a measurement of the fusion will be inaccurate. Hence the 
method of determining a three-dimensional relationship of probe spot signals within 
a region of interest (such as a nucleus or other region) will permit a large scale 

15 quantitative analysis of three-dimensional distances between any two probe signals. 
For example, the three-dimensional coordinates of a red signal are determined, and 
the three-dimensional coordinates of a green signal are determined, and the 
coordinates are then analyzed to determine if they are overlapping (suggesting a 
translocation that has moved them into contiguous genetic loci) or separate (on 

20 different genetic loci than they would be normally). 

EXAMPLE 1 
Device for Performing Automated FISH 

FIG. 4A shows such an automated spot counter 10 in accordance with the 
25 present invention. Briefly, the device 10 includes an automated optical microscope 
12 (such as a confocal microscope) having a motorized stage 14 for the movement 
of a slide 16 relative to the viewing region of the viewing portion 18 of the 
microscope, a camera 20 for obtaining electronic or digital images from the optical 
microscope, a processing system 22 for counting the spots, and a memory 24 and a 
30 high resolution color monitor 26 for the storage and display respectively of images 
processed by the device 10. 
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In a disclosed embodiment, the classification device 1 0 is automated and 
computer implemented, and therefore also includes, in addition to the motorized 
stage 14, an automated apparatus for focussing, for changing lens objectives 
between high and low power, and for adjustment of the light incident of the slide, as 
5 well as circuitry for controlling the movement of the motorized stage, typically in 
response to a command from the processing system. The microscope may also 
include an automated slide transport system for moving the slides containing the 
specimen to be classified on to and off of the motorized stage, and a bar code reader 
for reading encoded information from the slide. An example of a microscope 

10 performing at least some of these functions is manufactured by Carl Zeiss, Inc. of 
Germany, or Atto Instruments of Rockville, Maryland. In particular embodiments, 
the microscope is a confocal microscope from Atto Instruments, such as that shown 
in PCT WO 99/22261, a Laser Scanning Microscope LSM 510 from Carl Zeiss, 
Inc., or an Axioplan 2 microscope from Carl Zeiss, Inc., equipped with a CARV 

15 module available from Atto instruments. An example of a camera 20 suitable for 
use with the invention, is a Quantix CCD camera available from Photometries of 
Tuscon, Arizona. 

The signal counting device 10 is shown in FIG. 4B with particular emphasis 
on the classification elements embodied in the processing system 22. The 

20 processing system 22 may include an image processor and digitizer 42, and a 
general processor 46 with peripherals for printing, storage, etc. The general 
processor 46 can be an INTEL PENTIUM microprocessor or similar microprocessor 
based microcomputer, although it may be another computer-type device suitable for 
efficient execution of the functions described herein. The general processor 46 

25 controls the functioning and the flow of data between components of the device 10, 
may cause execution of additional primary feature signal counting algorithms, and 
handles the storage of image and classification information. The general processor 
46 additionally controls peripheral devices such as a printer 48, a storage device 24, 
such as an optical or magnetic hard disk, a tape drive, etc., as well as other devices 

30 including a bar code reader 50, a slide marker 52, autofocus circuitry, a robotic slide 
handler, the stage 14, and a mouse 53. Although a single processor system is 
shown, the invention could also be carried out in a variety of other systems. 
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The depicted devices include computer-readable media such as a hard disk to 
provide storage of data, data structures, computer-executable instructions, and the 
like. Although the description of computer-readable media above refers to a hard 
disk, other types of media which are readable by a computer, such as removable 
5 magnetic disks, CDs, magnetic cassettes, flash memory cards, digital video disks, 
and the like, may also be used. 

The image processor and digitizer 42 (hereinafter referred to as the image 
processor) digitizes images from the digital camera 20 and performs a primary 
algorithmic classification on the images to filter out unwanted information. The 

10 image processor 42 and the general computer 46 may each access read-only and/or 
random access memory, as would be readily apparent to one skilled in the art, for the 
storage and execution of software necessary to perform the functions described 
relative to that processing component. Further, each component 42 and 46 includes 
circuitry, integrated circuit chips, etc. for the control of communication or data 

15 transfer over the data bus 54, as well as other functions typical of similar processors. 

The steps performed by the computer implemented system are illustrated in 
_ FIG. 5A. The tissue is first prepared and subjected to FISH (60) in the conventional 
manner, for example, as described in Example 2. The system then obtains multiple 
confocal optical sections (62), and the successive images are transformed by 

20 detecting and representing positional values in each image of fluorescent emission of 
a particular color as an array of digital values. Localized increases (spikes) in the 
digital values may represent signals in the optical section (64). A histogram is then 
generated (66) which produces a saddle point (described in association with FIG. 8) 
that permits fluorescent intensity image components below a threshold value to be 

25 eliminated (68, 70). Among the remaining image components, those that are 

substantially contiguous are grouped into segments according to various criteria 
(72). Additional filtering can be performed (74), for example, to account for 
autofluorescence and spot clusters. 

With the spot signals then resolved in three-dimensional space, they are 

30 counted. Additionally, the counted signals can be combined into a single max 

image, which is similar to the two-dimensional view seen through a conventional 
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microscope. The max image is a convenient view for rapidly reviewing the FISH 
results in two dimensions. 

In two-color FISH, signals from a second probe (usually of a different color) 
may also need to be detected (76). In that event, an optical filter can be changed to 
5 detect the new color, or an incident beam of laser light of a different color or 

wavelength can be directed at the tissue section. The signals of a different color are 
then resolved in three-dimensional space using steps 62-74 as previously described. 
Once no more signals of a different color are to be obtained, a ratio of the signals 
from the test probe to the reference probe is calculated to determine whether there is 
10 a gene copy alteration. 

EXAMPLE 2 

Example of Automated FISH Signal Counting From a Series of Confocal 
Images in High Throughput Analysis with Tissue Microarrays 

Gene amplification is an important mechanism for the up-regulation of critical 
15 genes involved in cancer initiation and progression. A number of important 

oncogenes have already been found to be activated by DNA amplification. These 
include the HER-2 (17ql2), C-MYC (8q24), PRAD1/CYCLIN D (1 lql3), FGFR-1 
(8pl2), and FGFR-2 (10q24) oncogenes. All of these are examples of genes for 
which alterations in gene copy number could serve as an indicator or disease onset 
20 or progression. 

This example uses tissue microarrays, of the type shown in PCT publications 
WO US99/04000 and 04001 (which are fully incorporated by reference) as a high 
throughput technique for efficiently performing FISH on hundreds of tissue sample 
specimens. An example of such a tissue microarray is also shown in FIG. 12. In the 
25 absence of an automated technique for counting probe signals in the tissue 

microarray specimens, it would take many hours for each of the tissue specimens in 
the array to be examined and scored. However, using the automated technique 
described herein, hundreds or even thousands of the tissue specimens in the array 
can be examined and scored in much less time. 
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Prostate tumor microarray 

Formalin-fixed and paraffin-embedded tumor and control specimens were 
obtained from the archives of the Institutes for Pathology, University of Basel 
5 (Switzerland) and the Tampere University Hospital (Finland). The least 

differentiated tumor area was selected to be sampled for the tissue microarray. The 
specimens that were interpretable for at least one FISH probe included the 
following: I) transurethral resections from 32 patients with benign prostatic 
hyperplasia (BPH) to be used as controls; II) 223 primary tumors, including 64 

10 cancers incidentally detected in transurethral resections for BPH; 145 clinically 

localized cancers from radical prostatectomies, and 14 transurethral resections from 
patients with primary, locally extensive disease; III) 54 local recurrences after 
hormonal therapy failure, including 3 1 transurethral resections from living patients 
and 23 specimens obtained from autopsies; IV) Sixty-two metastases collected at the 

15 autopsies from 47 patients who had undergone androgen deprivation by 

orchiectomy, and had subsequently died of end-stage metastatic prostate cancer. 
Metastatic tissue was sampled from pelvic lymph nodes (8), lung (21), liver (16), 
pleura (5), adrenal gland (5), kidney (2), mediastinal lymph nodes (1), peritoneum 
(1), stomach (1), and ureter (1). 

20 

Construction and sectioning of tissue microarrays 

The prostate tissue microarray was constructed using a tissue arraying 
instrument that created holes in a recipient paraffin block, and acquired tissue cores 
from the donor block by a thin-walled needle with an inner diameter of 0.6mm, held 

25 in a X-Y precision guide. The cylindrical sample was retrieved from the selected 

region in the donor and extruded directly into the recipient block with defined array 
coordinates. A solid steel wire closely fit in the tube was used to transfer the tissue 
cores into the recipient block. After the construction of the array block, multiple 
Sum sections were cut with a microtome using an adhesive-coated tape sectioning 

30 system (Instrumedics, Hackefisack, New Jersey). H&E stained sections were used 
for histologic verification of tumor tissue on the arrayed samples. 
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Fluorescence in-situ hybridization (FISH) 

Two-color FISH to sections of the arrayed formalin-fixed samples was 
performed using a Spectrum Orange-labeled androgen receptor (AR) probe with 
corresponding FITC-labeled centromeric probes (Vysis, Downer's Grove, Illinois). 
5 The hybridization was performed according to the manufacturer's instructions. The 
following tissue treatment protocol was developed to allow formalin-fixed tumors 
on the array to be reliably analyzed by FISH: the slides of the prostate tissue 
microarray were first deparaffinized, acetylated in 0.2 N HC1, incubated in 1 M 
sodium thiocyanate solution at 80 degrees C for 30 minutes and immersed in a 

10 protease solution (0.5mg/ml in 0.9% NaCl) (Vysis, Downer's Grove, Illinois) for 10 
minutes at 37 degrees C. The slides were then post-fixed in 10% buffered formalin 
for 1 0 minutes, air dried, denaturated for 5 minutes at 73 degrees C in 70% 
formamide/2x SSC (SSC is 0.3M sodium chloride and 0.03M sodium citrate) 
solution and dehydrated in 70, 80, and 100% ethanol, followed by proteinase K 

15 (4^ig/ml phosphate buffered saline) (GIBCOBRL, Life Technologies Inc., 

Rockville, Maryland) treatment for 7 minutes at 37 degrees C. The slides were then 
dehydrated and hybridized. The hybridization mixture contained 3 pj.1 of each of the 
probes and Cotl-DNA (lrng/ml; GIBCOBRL, LifeTechnologies Inc., Rockville, 
Maryland) in a hybridization mixture. After overnight hybridization at 37 degrees C 

20 in a humid chamber, slides were washed and counterstained with 0.2 j^M DAPI. 
FISH signals were scored with a Zeiss fluorescence microscope equipped with a 
double-band pass filter using x40-xl00 objectives. The relative number of gene 
signals in relation to the centromeric signals was evaluated for a region of interest. 
Criteria for gene amplification were as follows: at least 3 times more test probe 

25 signals than centromeric signals per cell in at least 10% of the tumor cells. 

Test/control signal ratios in the range between 1 and 3 were regarded as low level 
gains, and were not scored as evidence of specific gene amplification. 

Output from the two color AR FISH is shown in FIG. 5B, which shows a series 
of optical sections 1-5, and a max image M that combines the fluorescent signal 

30 segments of the optical sections 1-5. A "signal segment" in an optical section, in 

combination with signal segments from other optical sections, are selectively chosen 
to make up a complete probe signal seen in the Max view. The red signal segments 
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in the optical sections (and the red signals in the Max view) have weaker signals 
than the green signals. Any of the two-dimensional optical sections 1-5, or the max 
image M, can be analyzed to determine the ratio of red to green signal segments or 
signals. As shown in FIG. 5B, the ratio of red to green (seen as a ratio of weaker to 
stronger spots) is substantially 1, throughout the optical sections 1-5 and in the max 
image M. This indicates that there is not an amplification of the AR, at least in the 
tissue section illustrated in FIG. 5B. In a gray scale version of FIG. 5B, the green 
signals appear as brighter spots, and the red signals are more difficult to detect. 

Alternatively, a three-dimensional representation of the max image can be 
obtained by analyzing the optical sections that contain each colored signal segment, 
and separating spot signals that overlap in an axial (vertical) or transverse 
(horizontal) direction. The separated signals can then be counted, and a ratio of red 
to green signals more accurately determined. Examples of FISH output from the 
green labeled probe, obtained by the three-dimensional analysis system of the 
present invention, are shown in FIG. 6A, which illustrates FISH-signal counting 
from a series of confocal images. Cuts 1 through 8 are optical sections obtained by 
the confocal microscope filtered to obtain only the green signal segments at different 
levels, while the Max-image is a combination of the cuts 1-8 (and is similar to the 
superimposed images that would be obtained in conventional two-dimensional 
FISH). The total number of separate green signal segments counted in cuts 1-8 was 
393 (summing the number of signal segments shown in brackets above each section 
number 1-8). After automated analysis of the signal segments, with separation of 
vertically and horizontally overlapping signals, it was determined that there were 
283 unique signals. In a gray scale version of FIG. 6A, the green signals appear as 
slightly darker spots, such as the one jutting atop cut #6. 

EXAMPLE 3 
S6K Amplification in Breast Cancer 

In this example, the biological consequences were examined of genomic 
rearrangements at 17q23, a locus amplified in up to 20% of primary breast cancers 
as assessed by comparative genomic hybridization (CGH). An array of primary 
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breast cancers was constructed, and used to determine S6K gene amplification 
frequencies in vivo. 

Two cohorts of primary breast cancers were studied using the tissue microarray 
analyses. The first microarray consisted of 372 ethanoKfixed primary breast 
5 cancers. The second microarray consisted of 612 primary breast cancers from the 
years 1985-1995, from patients with complete clinico-pathological information, 
including an average of 5.4 years of follow-up. Both series of tumors were 
analyzed, with 668 cases being informative for all experimental and clinical data. 
Both tumor cohorts were obtained from the Institute of Pathology, University of 

10 Basel. The tumor samples included 73.3% ductal, 13.6% lobular, 3% medullary, 
2.6% mucinous, 1.5% cribriform, 1.4% tubular, 1 . 1 % papillary carcinomas, 1.9% 
ductal carcinoma in situ, and 1 .7% of other rare histological subtypes. The grade 
distribution was 24% grade 1, 40% grade 2, and 36% grade 3. The pT stage was pTl 
in 32%, pT2 in 51%, pT3 in 7%, and pT4 in 10%. 

15 A SpectrumOrange labeled PAC probe specific for S6K and a SpectrumGreen 

labeled chromosome 1 7 centromere probe (Vysis, Downers Grove, IL) was used for 
^,copy number analysis. Interphase FISH to breast cancer cell lines was done as 
previously described in Barlund et al., Genes Chrom. Cancer 20:372-376, 1997. 
The hybridizations are evaluated using a Zeiss confocal fluorescence microscope, 

20 and following the algorithm shown in FIG. 5A. 

The breast cancer tissue section showed a high-level SK6 gene amplification, 
with a higher number of red signals than green reference signals, as shown in FIG. 
6B. In a gray scale version of 6B, the red signals appear as brighter dots, such as the 
loose cluster of dots in the center of image 5. 

25 

EXAMPLE 4 

Graphical User Interface for Three-Diinensional FISH Signal Counting 

Although the invention can be implemented in a variety of computing 
environments, the following examples are implemented in MATLAB, which is 
30 available from Mathworks of Natick, Massachusetts. FIG. 7 shows a graphical user 
interface (GUI) that is displayed by the system for processing the fluorescent images 
and counting signals associated with probes that have hybridized to a target nucleic 
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acid sequence. The image- displayed on the interface is an image of one of the 
optical sections of a tissue section that has undergone FISH (in this case slice=5 
refers to the 5 th of 8 contiguous optical sections at successively deeper depths of the 
tissue section; stack=7 refers to the number of the tissue section on the tissue array 

5 that is being processed; and threshold=26 refers to a threshold value of signal 
intensity below which intensity values are eliminated from calculations). 

FIG. 8 is a histogram which illustrates how the threshold value (threshold=26) 
is determined in this example. The histogram plots the relative frequency of gray 
levels (which corresponds to signal intensity) across the pixels of the image of each 

10 optical section, where the x-axis is the frequency of occurrence, and the y-axis is a 
brightness value. This graph shows two modes, in which the first mode (the sharp 
spike) represents noise (primarily dark background), and the second mode (the 
broader peak) represents useful signals (such as brighter pixels associated with 
fluorescent intensity). To determine a saddle point (threshold value), the histogram 

15 is smoothed, and the derivative of the resulting graph is calculated. The point T=26 
is the point at which the derivative of the curve changes value from + to and this is 
the value that is selected as the threshold. Pixels having a brightness below this 
level are eliminated from further data manipulation. 

FIG. 9A shows an image which is displayed of contiguous signal segments in 

20 different optical sections 1-8 of a tissue stack (which corresponds to a microarray 

spot). This FISH signal is designated spot 224, and the image intensity of the signal 
segments that are present on optical sections 1-8 can be seen in the small image 
panels across the bottom of the display. Each small panel 1-8 is a section of the 
three-dimensional space within the tissue section, and the number of pixels where 

25 signal is present is shown by the white boxes in the display panels. Although 

several three-dimensional representations of the signal segments are shown in FIG. 
9, the center representation (with segments labeled la-8a) will be used for purposes 
of illustration to explain how the different signal segments are combined into a 
single spot signal, or how certain signal segments are discarded. 

30 The relatively small brightness signals (which correlate with respectively few 

white pixels) in panels 1, 2, 3 and 4 are mapped into correspondingly small three- 
dimensional geometric boxes la, 2a, 3a and 4a, having a volume proportional to the 
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area of the illuminated pixels in the panels 1-4. The absence of any illuminated 
pixels in panel 5 corresponds to an absence of any geometric figure in space 5 a (or 
can indicate that any pixels were below a threshold value and therefore not 
considered for further data manipulation). The area of the white pixels in panel 6 
5 correspond to the volume 6a, and the area of the white pixels in panel 8 corresponds 
to the volume 8a, while the very large illuminated area in panel 7 corresponds to the 
figure 7a, which has the largest volume of any of the geometric figures. 

One function of the algorithrn is to eliminate background noise from actual 
probe signals. This can be done by setting a threshold value, below which a signal 

10 segment can be considered not to be a valid signal segment. The signal segments in 
panels 1, 2, 3, 4 and 6 do not meet the threshold criteria for minimum illuminated 
pixels, and are therefore not considered when combining the signal segments into an 
actual signal. The signal segments in panels 7 and 8 do, however, meet the criteria 
for minimum pixel illumination. Since more pixels are illuminated in panel 7 than 

15 in panel 8, the location of the signal is assigned to level 7, although alternatively it 
could be considered to reside in both levels 7 and 8. 

Other signal overlap situations that the algorithm can resolve are also shown 
schematically in FIGS. 9B and 9C. In FIG. 9B a side view of optical sections 1-10 
is shown, in which two spot signals vertically (axially with respect to the microscope 

20 lens) overlap one another. The signal segments in sections 1, 5, 6 and 10 fall below 
a pre-selected threshold value, and are eliminated from further consideration. The 
remaining signal segments in sections 2, 3, 7, 8 and 9 are above the threshold, and 
are grouped into a first spot signal at contiguous sections 2-4, and a second spot 
signal at contiguous sections 7-9. The probable location of the top spot signal can 

25 be assigned to section 3, where the strongest signal segment of the top spot signal is 
found, or alternatively the spot signal can be considered to reside at sections 2-4. 
The probable location of the bottom spot signal can be assigned to section 8, where 
the strongest signal segment of the bottom spot signal is found, or in sections 7-9 
across which the signal segments extend. 

30 FIG. 9C schematically illustrates how vertically and/or horizontally 

(transversely with respect to the optical axis of the microscope lens) overlapping 
spot signals can be resolved. A side view of optical sections 1-8 is shown in which 
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the signal segments in sections 1 and 8 do not meet a threshold, and are eliminated. 
In section 4, there are two horizontally non-contiguous signal segments. The signal 
segment 4B (represented by a gray bar) does not meet a threshold value and is 
eliminated, while the segment 4A (represented by the black bar) does satisfy the 
5 threshold and remains for consideration. Similarly, there are two non-contiguous 
signal segments in section 5. The segment 5B (represented by the gray bar) meets 
the threshold, while the segment 5 A (represented by the black bar) does not meet the 
threshold and is eliminated. Once the signal segments 4B and 5A are eliminated, the 
segments can be resolved into a top spot signal (comprising segments 2, 3 and 4A) 

10 and a bottom spot signal (comprising segments 5B, 6 and 7) 

FIG. 10 illustrates a three-dimensional representation of a tissue section that has 
been subjected to FISH and analyzed by the method of the present invention. The 
three-dimensional space is divided into x-y-z coordinates, in which the z axis is 
associated with successive optical sections 1 -8 of the tissue section, and the signal 

15 segments of all of the signals in the section are illustrated. Signal segments are 
illustrated as small black cylinders, and the assigned location of each spot is 
illustrated as a gray colored larger cylinder, some of which are designated Al, A2, 
A3, A4, A5, A6, A7 and A8. Each of these gray cylinders A is associated with a 
corresponding sphere B in the max plane, in which the volume of each sphere B can 

20 be proportional to the volume of the A cylinder with which it is associated. 

The system additionally provides an opportunity for a user to provide 
guidance during spot counting. For example, the user can specify a particular area 
of interest by selecting it on the screen. Typically, the max image (e.g., image M of 
FIG. 5B) is presented, and the user may select an area or areas via a pointing device 

25 (e.g., a mouse). Counting is then limited to only the selected area or areas. Such a 
feature can be particularly useful when the user recognizes that a certain area of the 
image relates to a region of interest. 

The system also provides a way to eliminate a specified area or areas 
selected via a pointing device (e.g., a mouse). Portions of the image within the 

30 specified area or areas (sometimes called "gated areas") is ignored when spots are 
counted. 
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EXAMPLE 5 
Algorithm Overview 

The method of the present invention facilitates the analysis and quantitation 
of FISH on tissue microarrays, by overcoming previous obstacles to automation. 

5 This method, as implemented in the algorithm of this example, applies dual-color 
FISH with different gene probes and corresponding chromosomal reference probes 
to tissue microarrays. Because each tissue section is a three-dimensional volume, a 
confocal microscope system is used to generate a three-dimensional stack of a 
number of fluorescent images (e.g., 24 images) over defined tumor areas of each of 

10 the 4 um thick tissue specimens on the tissue microarrays. The algorithm identifies 
signal segments on each level along the Z-axis, and differentiates signal segments 
which overlap vertically above the X-Y plane. 

The algorithm can be refined to account for spot clusters, and a variety of 
filtering mechanisms can be used to remove signals related to, for example, auto- 

15 fluorescence and other false signals. A variety of parameters used during the 
algorithm can be modified by the user to account for circumstances related to a 
particular image or set of images. For example, since the magnification used for an 
image can vary, the parameters of the algorithm can be modified to account for the 
relative size of a pixel. There are many other examples of parameters (e.g., numeric 

20 values) that can be varied to avoid a strict adherence to a particular value. Various 
results of intermediary calculations are stored to facilitate efficient re-calculation 
when parameters are modified by the user. 
i An overview of an implementation of the algorithm is shown in FIG. 13. 

Typically, the algorithm is provided with a set of digital image slices (e.g., a raster 

25 image) representing a set of observations (e.g., of a biological specimen subjected to 
FISH) taken at different depths along a z-axis via a confocal microscope. Thus, the 
set of images is sometimes called a "stack." The stack typically consists of 
anywhere from 8-32 images, although more images may be used. Satisfactory 
results have been obtained with 24 images. 

30 At 1302, possible fluorescent image components are identified in the slices. 

For example, pixels meeting certain criteria are selected as possibly corresponding 
to a location within or near a spot and placed into a set of binarized images. The 
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criteria can include an intensity level, and the images may be subjected to various 
filters before constructing the binary image. 

At 1304, the resulting pixels in each slice are projected onto a projection 
image. For example, a binary projection image can be constructed from a stack of 
5 binary images by setting pixels in the binary projection image at locations (X,Y) 
whenever there is any pixel set in any of the stack of binary images at the same 
(X,Y) location (regardless of the Z location). 

At 1306, insignificant regions in the image slices are discarded. So, for 
example, single lone pixels in a slice, or a contiguous set of 2-3 pixels are ignored, 
10 removed, or labeled as insignificant. A similar process can be applied to the 
projection image. 

At 1308, each contiguous region in the projection image is considered in 
turn. For example, minimal rectangles can be drawn around each substantially 
contiguous region in the projection image, numbered, and considered in turn. 
15 At 1 3 10, contiguous regions in the image slices associated with the 

projection image region under consideration are analyzed and grouped into spot 
candidates. At 1320, the next region in the projection image is considered, until 
each has been considered. 



20 example, certain candidate spots may be false spots, or others may actually be 
clusters representing plural spots. 

Finally, at 1324, the spots are counted. The spot count can then be used vis- 
a-vis other spot counts to provide a ratio (e.g., a ratio of genes to centromeres). 



In this method, the traditional signals-per-nucleus approach is replaced by an 
overall gene-to-chromosome ratio in a defined tumor area. Therefore, the ratio 
between the signals of gene probes and the reference probes are calculated 
30 irrespective of the number of nuclei. This new method for automated capturing and 
quantitation of FISH signals significantly advances the utility of tissue microarrays 



At 1322, filtering an augmentation can be applied to the candidate spots. For 
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EXAMPLE 6 



Details of Algorithm 
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as a high-throughput tool for detection of relative gene copy number alterations 
during cancer development and progression. 

Although the invention can be implemented in a variety of computing 
environments, the following examples are implemented in MATLAB, which is 

5 available from Mathworks of Natick, Massachusetts. Various MATLAB-based 
Graphical User Interface (GUI) tools allows visualization of three-dimensional 
shapes of spots as well as their two-dimensional projections on the slices. The 
dynamic interface provides for manipulation with many input parameters for the 
threshold, number of slices, and filtering. It also provides for visualization of 

10 differently colored spots, storage, displaying and printing data in the form of 
different 3-D images, diagrams, and tables. 

An implementation of the algorithm in a scenario with 24 image slices can 
have the following features: 

(1) A morphological top-hat transform is applied to each of the 24 images to 
15 yield 24 outputs, each possessing brightness intensity spikes jutting above an 

essentially flat background. Each bright spike can correspond either to a signal 
segment, or to noise. 

(2) Each top-hat image is thresholded to produce a stack of 24 binary images 
showing spike locations. Morphological filters are applied to the binary images to 

20 eliminate noise, and touching spots are segmented. 

(3) Binary spot markers occurring as vertical neighbors in the stack are 
grouped into one final spot located at a particular assigned stack level. Various 
parameters of the algorithm are set to fit the physical characteristics of the images. 
These include window size for the top-hat transform, threshold levels, and sizes of 

25 filter structuring elements. 

The performance of the algorithm can be based on the following main steps: 
1. Given L + I slices of the fluorescent images, X 0 , Xj, X L , of sizes NxM, 
calculate the max-image F max : 
30 r max (./,/) = max{X k (jJ);k -0,1,...,/.}. 
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The histogram, H(t) 9 of this image is calculated and smoothed, and then the 
approximate value of the threshold, T, is determined as the minimal saddle-point 
between two modes of the histogram. The histogram is displayed on the screen and 
the threshold can be changed on-line. The chosen value- of the threshold is used in 
the next steps for computing binary images. 
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2. For each image X ky k=0, 1,...,L, the top-hat transform is calculated. 
Sometimes the top-hat transform^ is called "image minus opening." The calculation 
is 



Y k = X k -X k oB, 



(1) 



where X k ° B is opening by the structuring element B , which is taken to be a circle 
fitting within a 7x7 square (roughly, a circle with a diameter of 7). Other structuring 
15 elements can be used (e.g., the following examples show a 5x5 square) and can be 
specified by the user via a user interface feature. The opening is defined by 
XoB = (X0 B) © B , and calculated as follows: 



(X0B)(j\O = mm{X(j + j x ,i + /* );(_/,/) e B), 
20 {X © 5)0', i) = max{X(y + j\J + i.fctAO-e B}. 



(2) 
(3) 
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To calculate the top-hat transform in (1), the decomposition of the transform 
by two top-hat transforms with 1x5 and 5x1 structuring elements is used, which 



results in the fas 



X0 



performance of the transform 5x5: 
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®[l 1 1 1 l]. (5) 



The data of the top-hat transform Y k is stored for future processing in case the value 
of the threshold needs to be changed. 

5 

3. Each top-hat transform Y k is thresholded by the value T, resulting in a 
binary image B k , k=0,l,...,L, 

5,(7,0 = 1, if Y k (JJ)>T, 
10 5,(7,0 = 0, if Y k (j,i)<T. 

A projection of multiple slices of the image can be calculated as 

5 max (7\0 = l> if 5,(7,0 = 1, 

15 

for at least one k. Typically, the projection is calculated with reference to all 
available image slices, in which case the binary image 5 max is sometimes called the 
"projection binary image" because it is roughly equivalent to peering down through 
the stack of binary images. In some sense, the binary image B max is also a 
20 "maximum" image; however, in the example, the maximum is computed comparing 
only l's and 0's. 

4. The obtained binary images 5*, are processed to find the separate regions 
revealed by the thresholded top-hat transform. Each actual spot is composed of 
25 some number of these regions on different binary slices. Each of these slice regions 
is referred to as a signal segment, so that each spot signal is composed of a union of 
signal segments. The software can identify signal segments by locating substantially 
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contiguous groups of pixels, number the signal segments, and calculate the 
following data: 

A. coordinates of the signal segment center; 

B. the number of pixels composing the signal segment; 

5 C. the coordinates of the minimal rectangle which contains the signal segment. 

Signal segments composed of a single pixel are ignored during this procedure but 
their presence can be used during other steps. 

5. The projection binary image 5 max is processed similarly, and its separate 
10 regions (projection binary image regions) are identified by locating substantially 
contiguous groups of pixels and numbered. The following are calculated for spot 
signals on the projection binary image: 

A. coordinates of the projection binary image region center; 

B. the number of pixels (Ts) composing the region; 

15 C. the coordinates of the minimal rectangle which contains the region; 

D. the number of signal segments from the different optical sections, which lie 
inside the surrounding rectangle of the region; and the largest such signal 
segment is marked. 

The algorithm then attempts to identify each of the spots by integrating the 
20 signal segments into spot signals via analysis of the sizes, intensities, and locations 
(in 3 dimensions) of the signal segments. Typically, for purposes of determining 
whether a particular signal segment is of sufficient size to indicate a contiguous 
connection, a threshold (e.g., 3 pixels) is useful to designate as a cutoff. So, for 
example, signal segments having 3 or fewer pixels can be ignored when determining 
25 whether the slice above and/or below a signal segment is contiguous with the 
particular signal segment. 

Specifying such a threshold can be particularly useful in resolving closely- 
neighboring spots, such as two spots having substantially similar (X,Y) coordinates 
but appearing in different slices of the image. Examples of such scenarios are 
30 shown in FIGS. 9B and 9C. As shown, there can be more than one spot in a vertical 
direction or in a horizontal direction. 
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During processing, segments that are substantially vertically contiguous can 
be associated with a particular optical slice having the greatest signal intensity, 
which is often the center of a single spot. However, it may be that there is another 
image slice with great signal intensity in the same segment, in which case there may 
be more than one spot in the segment. 

Typically, the information about a spot candidate includes a size (e.g., total 
number of pixels across the image slices) and an intensity (e.g., the brightest pixel in 
the region of the slice having the most pixels for the spot candidate). 

A data flow diagram for an implementation of the algorithm is shown in FIG. 
14. The slice data 1402 is used to construct a max image 1404 and the top-hat slice 
data 1406. The max image 1404 is useful for presenting to the user to give the user 
an informative depiction of a combination of the image slices and for various user 
interface features. It can be constructed by taking the pixel with the greatest 
intensity out of the image slices in the z-plane. 

The top-hat slice data 1406 can be generated as described above. Max top- 
hat image 1408 can be generated by finding the greatest intensity of the top-hat 
.slices for each (X,Y) location. The max top-hat image 1408 need not be displayed, 
but is useful, in combination with the top-hat slice data 1406 to construct the 
histogram 1410. 

A threshold 1412 is chosen with reference to the histogram 1410 as 
described above. Then, the thresholded slice data 1420 can be constructed by 
generating a binary image for each of the slices, where pixels in the slices are set if 
the slice image in the slice data 1402 exceeds the threshold 1412. 

From the thresholded slice data 1420, substantially-contiguous regions of the 
thresholded slice data can be identified to generate regions 1422. The regions are 
filtered to produce filtered regions 1424 of the thresholded data. 

Also from the thresholded slice data 1420, a projected binary image 1426 can 
be constructed, and substantially-contiguous regions 1430 within the projected 
binary image 1426 found. These regions can be filtered to generate filtered regions 
1440 of the projected image. 

Z-axis analysis data 1428 can be generated with reference to the filtered 
regions 1424 and the regions 1430 and used to generate the candidate spots 1432, 
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from which, false spots 1434 and clusters 1442 can be identified. Additional 
filtering can be performed to generate spot count 1444. Finally, display data 1450 
can be used to present graphical results to the user. 

The data flows in the diagram are general only, -and often data in upstream 
5 data components can be relied upon by downstream data components (or vice versa), 
even if not specifically shown in FIG. 14. As the data components are generated, 
they are often saved to permit efficient re-calculation when a parameter is adjusted 
to account for a particular image scenario. 

In addition to the above-described filters, additional filtering mechanisms 

10 can be provided to improve accuracy of spot counting. For example, one feature 
relates to removing false spots relating to, for example, autofluorescence. Two 
separate approaches relate to removing based on intensity and intensity in 
combination with area. For example, after candidate spots are identified, an 
intensity of the spot can be calculated. If the intensity exceeds a certain threshold 

15 (say, two standard deviations above the mean intensity), the spot can be discarded 
and not counted. In addition, or alternatively, the intensity can be combined with 
(e.g., multiplied by) the area or volume of a spot. Again, if the value is over a 
particular threshold, the spot can be discarded and not counted. The approaches can 
be combined. 

20 For example, the image in FIG. 1 5 A illustrates spot candidates in an image 

of normal prostate tissue in which the X centromere has been labeled via FISH. 
Some of the spot candidates are actually autofluorescent tissue, and can be 
eliminated from the spot count. The image can be presented as part of the user 
interface. 

25 The graph in FIG. 15B shows the intensity of the 149 spot candidates 

identified in the image. A mean and standard deviation of the spot candidate's 
intensities is calculated, and used as a threshold (e.g., two standard deviations above 
the mean). Spot candidates having intensities above the threshold are discarded 
(e.g., identified as "small autofluorescent tissue particles 1 '). 

30 The graph in FIG. 15C is similar to that of 15B, but also includes an area 

component (e.g., by multiplying the intensity by the number of pixels" and not 
included in the spot count). Again, spot candidates having a rating above the 
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threshold are discarded (e.g., identified as "large autofluorescent tissue particles" 
and not included in the spot count). Eight candidates can be removed because they 
exceed the threshold. The threshold in either case can be adjusted manually by a 
user if the default setting is inappropriate. 
5 FIG. 15D shows the image of 15 A, but the small autofluorescent tissue 

particles, large autofluorescent tissue particles, and the true FISH signals are 
differentiated by presenting each as a different color. The image can be presented as 
part of a user interface for the system. Eliminating the autofluorescent particles 
typically leads to more accurate spot count results. In a color version of the image, 

10 the image components 1510, among others, are portrayed in the color green to 

indicate that they are considered true FISH signals. The image components 1520 
and 1 521 , among others, are portrayed in the color yellow to indicate that they are 
small autofluorescent tissue particles. The image components 1530-1533, among 
others, are portrayed in the color blue to indicate that they are large autofluorescent 

15 tissue particles. A variety of other colors or other ways of emphasizing and 
differentiating the image components related to autofluorescence can be used. 

Still another feature than can be implemented is removal of spots appearing 
at substantially identical positions in two channels (e.g., red and green channels 
generated in two separate images, such as an image for a test probe and an image for 

20 a reference probe). Typically, such a situation means the spot can be ignored (e.g., 
not counted for either image). 

Yet another feature relates to identifying clusters and estimating the number 
of spots in a cluster. In some cases, certain criteria (e.g., the size of a projection 
binary image region or the total number of pixels in slice regions related to a 

25 projection binary image region) indicates that a cluster of spots is present. And, the 
software can be configured to accept user guidance as to which portions of the 
image are a cluster (e.g., via selection by pointing device). Determining the number 
of spots in a cluster can be difficult; however, one way of providing accurate 
estimates is to provide a calibration mechanism. 

30 For example, in one instance, 200 sets of clusters were analyzed manually to 

determine how many spots were in the clusters. These clusters were then subjected 
to the above-described algorithmic analysis. Under certain circumstances, a factor 
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of 2.5 was determined to be appropriate for determining how many spots were in a 
cluster. Thus, for example, if an area is identified as a cluster and the standard 
algorithmic analysis shows 4 spots, an estimate of 10 is provided. Other, more 
complex, analysis can be done, such as, for example, counting the total number of 
5 pixels associated with the signal segments related to a projection binary image 
region. 

For clusters, a mapping between spots detected and actual spots can be used. 
Sometimes detection of a small number of spots (e.g., 3) may in fact indicate a tight 
cluster of (e.g., 9) spots, while detection of a larger number of spots (e.g. 7) may be 

10 a loose cluster of (e.g., 9) spots. Therefore, a uniform gain factor is not appropriate. 
The appropriate method for handling clusters may vary depending on the target 
being counted. For example, certain genes have a higher propensity for clustering 
than do centromeres. The user can manually adjust the cluster detection and 
counting parameters manually. 

15 The number of spot signals is counted and the results are provided to the user. 

The interactive options of the program allow the threshold T to be changed, and 
process the spot counting of the fluorescent images from step 3, avoiding the 
repeated calculation of the top-hat transforms. 

20 The output data using this algorithm can include: 



2. 



1. 



Original images of all slices. 
Max-image. 



3. 



Histogram of Max-image. 
Max-image with colored spots. 



4. 



25 



5. 



Top-Hat transforms of the images. 



6. 



Number of spots. 

Max-image with colored spots after filtering out spots with less than some 
specified number of pixels. 



7. 



8. 



Original images with colored spot-cuts. 

Images with colored rectangles surrounding the spots. 

3-D view of each spot with sections lying on slices by vertical. 

2-D view of spot composition by projections. 
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9. 



10 



11 
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12. 3-D view of a number of spots with their projections by slices. 

13. Table of data for spot-cuts for L slices, which consists of: 



# spot-cut 


# slice 


^center 


y center 


# pixel 


Xle/l 




ybottom 




r.v.l. 


24 


4 


160 


124 


24 


160 


164 


120 


125 


1 



5 where r.v.l. is the abbreviation of relative vertical location, jc /e// , x righh y boUom , and y t 
are the coordinates of the rectangles surrounding the spot-cuts. The last box 
indicates whether the spot-cut is below, on, or above the true spot. 

14. Table of data for spots on the Max-image, which consists of: 



# spot-cut 


•^center 


y center 


# pixel 


X/eft 


Xrighl 


ybottom 


yiop 


224 


182 


164 


42 


180 


186 


160 


167 



10 EXAMPLE 7 

Scatter Plot of Spot Candidates and Calibration Features 

An additional user interface can be presented for assisting in analyzing and 
manipulating the data related to FISH experiments. For example, FIGS. 16A and 
16B show scatter plots generated for a FISH experiment in which the presence of 

15 centromere 17 and gene HER-2 were detected to generate a ratio indicating possible 
amplification of HER-2. 

Spots relating to centromere 1 7 typically do not cluster and are shown in 
FIG. 16A. Spot candidates are represented by appearing in a position appropriate 
for their intensity and size (e.g., number of pixels). The user can set a threshold 

20 parameter (e.g., 4) so that candidate spots having fewer than the specified number of 
pixels are excluded from the spot count. Also, a minimum intensity parameter can 
be set. 

Spots relating to HER-2 are shown in FIG. 1 6B and are more difficult to 
count due to potential clustering. For purposes of illustration, areas of the plot 
25 roughly relating to background noise 1652, real signals 1654, and clusters 1656 are 
circled. To facilitate determining an appropriate threshold, the user is presented 
with a calibration feature. An image (e.g., the maximum image 1404) is presented 
to the user, who can then verify that certain spot candidates are indeed spots, 
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although there are of minimal intensity. The user selects the spot candidate shown 
and activates a user interface element indicating the spot candidate is of minimal 
intensity. 

Subsequently, when the scatter plot 1650 of FIG. 16B is shown, the spot 
5 candidates of minimal intensity are shown in red to facilitate choosing an 
appropriate minimum intensity threshold (e.g., by adjusting an appropriate 
parameter in the algorithm), below which spot candidates are not counted as spots. 
Thus some of the points 165 8, typically near the bottom of the plot, are red. The 
minimum intensity threshold is typically adjusted to barely include the minimum 
10 intensity candidates. 

Also useful is the illustration of clusters 1656, which can be used to adjust 
the cluster selection criteria. Additionally, the user can at any time click on a 
particular spot candidate (e.g., 1660) to navigate to information about the spot 
candidate, including a three-dimensional view of the spot candidate. At the user 
15 interface, the user can then manually designate the spot candidate as a cluster or 
non-cluster. 

The scatter plot 1650 thus presents a useful tool for allowing the user to more 
easily grasp the totality of the image data and easily navigate to appropriate user 
interface screens for analyzing and adjusting the data, as well as calibrating the 
20 system. 



The system can also provide a user interface by which the user can guide the 
selection of areas to be analyzed. Although the system typically discards clearly 
25 aberrant data (e.g., a substantially total black or high-intensity image slice), the 

system may benefit from assistance from the user in determining which areas of an 
image are of interest or which are not. As described below, the user can select an 
area to be counted separately or an area not to be counted. 



30 areas of interest. Typically, the user interface 1702 presents a max image (e.g., 
1404). By using a pointing device (e.g., a mouse or trackball), the user can 
designate a particular area 1 706 as one to be processed separately. Multiple areas 



EXAMPLE 8 



User Direction of Areas to be Analyzed 



FIG. 17 shown a user interface 1 702 presented to the user for designating 
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can be added to an area set and processed separately. In this way, the user can direct 
the system to focus on particularly relevant portions of the image. 

The system also supports designating areas as not to be processed or 
included in the spot count. The algorithm can store the areas in a list and, for 
5 example, not count spots at a location (e.g., X, Y coordinates) within the area(s) 
designated. 

The above feature is useful because areas of the image can sometimes be 
identified as debris, stroma, connective tissue, or blood vessels, which tend to 
interfere with determining an appropriate spot count in certain scenarios (e.g., some 
10 material may be particularly prone to autofluorescence). 



15 evaluation and manipulation of the image data. FIG. 18 shows a user interface 1800, 
which depicts a spot candidate in three views, 1802, 1804, and 1806. The views 
assist in determining, for example, whether the spot candidate is a cluster, as is 
likely the case in the illustrated example. The view 1 806 is additionally processed to 
give a smoother representation of the spot candidate. The horizontally-contiguous 

20 region with the greatest area (e.g., number of pixels) is specially identified in the 

views with emphasis 1808. A status line 1818 indicates information about the spot 
candidate. 

A strip along the interface indicates the max image, and the 1 5 image slices 
(numbered 0-14) in binary form after thresholding. The image slice having the 

25 greatest area region is noted at 1820. User interface controls 1832 and 1834 allow 
navigation to spot candidates and identified clusters, respectively. 

In view of the many possible embodiments to which the principles of the 
invention may be applied, it should be recognized that the illustrated embodiments 
are examples of the invention, and should not be taken as a limitation on the scope 

30 of the invention. Rather, the scope of the invention is defined by the following 
claims. We therefore claim as our invention all that comes within the scope and 
spirit of these claims. 



EXAMPLE 9 



User Interface Including Three-dimensional Representations 
of Spot Candidate 
Still another user interface feature can be presented by the system to assist in 
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We claim: 

1 . A method for counting nucleic acid probe signals in a region of 
interest in a biological specimen, the method comprising: 

counting a number of test signals from a test probe; 

counting a number of reference signals from a reference probe; and 

determining a ratio of the counted test signals from the test probe to the 

counted reference signals from the reference probe, wherein the region of interest 

comprises multiple cells. ^ 

2. The method of claim 1 , wherein the reference probe is a 
polynucleotide that hybridizes to a centromere, and the number of reference signals 
from the reference probe approximates a nucleus count in the biological specimen. 

3. The method of claim 1 , wherein the reference probe recognizes a 
target on a same chromosome as the test probe. 

4. The method of claim 1 , wherein the test probe is a polynucleotide that 
hybridizes to a target sequence in a gene, and the reference probe is a polynucleotide 
that hybridizes to a reference sequence. 

5. The method of claim 3, wherein the reference probe recognizes a 
centromere of the same chromosome on which the gene of interest is contained. 

6. The method of claim 1 , further comprising obtaining successive 
images of the region of interest to distinguish overlapping signals in the biological 
specimen. 

7. The method of claim 6, wherein the successive images are optical 
sections of the region of interest. 

8. The method of claim 7, wherein the optical sections are at different 
depths of the biological specimen. 

9. The method of claim 8, wherein the successive images are 
transformed into digital representations in which contiguous signal segments in 
successive optical sections are combined into a single signal in a particular optical 
section in which a strongest signal segment is located. 

10. The method of claim 6, wherein different successive images are 
obtained for the test probe signals and the reference probe signals, and a quantity of 
test probe signals and.reference probe signals are determined. 
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1 1 . The method of claim 6, wherein successive images are obtained 
which show distinguishable test probe signals and reference probe signals, and a 
quantity of the test probe signals and reference probe signals are determined. 

12. The method of claim 6, wherein the successive images are obtained 
5 by confocal microscopy. 

13. The method of claim 1, wherein the ratio of signals is determined 
without reference to boundaries of a cell nucleus. 

14. The method of claim 1, wherein the ratio of signals is determined 
without reference to the boundaries of a cell. 

10 15. A method of counting visible signals from probes used with in situ 

hybridization of biological samples, the method comprising: 

obtaining a plurality of images at different levels of the biological sample; 
constructing a three-dimensional image indicating discrete signals at 
different levels of the three-dimensional image; and 
15 counting the discrete signals at different levels of the three-dimensional 

image. 

16. The method of claim 15, wherein the three-dimensional image is 
constructed by determining a location of a signal segment in the different levels of 
the biological sample, combining overlapping signal segments in contiguous levels 

20 into a single spot signal, and separating signal segments in non-contiguous levels 
into different spots. 

17. The method of claim 16, wherein the location of signal segments is 
determined by the presence of an increase in brightness intensity that indicates an 
increase of signal as compared to a background signal. 

25 18. The method of claim 1 7, wherein the probes display fluorescent 

signals, and the increase in brightness intensity is associated with an increase in 

fluorescence compared to the background signal. 

19. The method of claim 1 5, wherein the signals comprise test signals 

from a test probe and reference signals from a reference probe. 
30 20. The method of claim 1 9, wherein the test probe recognizes a gene of 

interest, and the reference probe recognizes a chromosomal locus having an 

expected quantity in the biological specimen. 
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21 . The method of claim 20, further comprising determining a ratio 
between the test signals and the reference signals. 

22. The method of claim 21, further comprising determining: 

(a) whether there is an increase in an expected ratio between the test 

5 signal and the reference signal, indicating an amplification of the gene of interest; or 

(b) whether there is a decrease in the expected ratio between the test 
signal and the reference signal, indicating relative loss of the gene of interest. 

23. The method of claim 19, wherein the test probe is selected from the 
group consisting of probes that recognize genes implicated or suspected in the 

10 development or progression of a tumor. 

24. The method of claim 15, wherein the biological samples are in a 
microarray. 

25. The method of claim 24, wherein the microarray comprises a tissue 
microarray. 

15 26. The method of claim 25, wherein the tissue microarray comprises 

tissue samples of a same tissue type taken from a plurality of donor specimens. 

27. The method of claim 15, wherein the plurality of images consists of 
between eight and thirty two images at different levels of the biological sample. 

28. The method of claim 1 5, further comprising: 

20 avoiding counting discrete signals having intensities exceeding a threshold 

intensity. 

29. The method of claim 15, further comprising: 

avoiding counting discrete signals having a combined intensity and area 
exceeding a threshold value. 
25 30. The method of claim 1 5, further comprising: 

avoiding counting discrete signals related to autofluorescent material. 
3 1 . The method of claim 1 5, further comprising: 

depicting a two-dimensional image representing the three-dimensional image 
for consideration by a user. 
30 32. The method of claim 3 1 , further comprising: 

emphasizing discrete signals related to autofluorescent material in the two- 
dimensional image. 
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33. The method of claim 15, further comprising: 
identifying a set of one or more discrete signals as a cluster; and 
counting the cluster as a number of discrete signals greater than the number 

9 

of discrete signals in the set. 

34. The method of claim 33 wherein the cluster is counted as a number of 
discrete signals indicated by applying a mapping to the number of discrete signals in 
the set. 

35. The method of claim 33 wherein the cluster is counted as a number of 
discrete signals indicated by a function calibrated via manual counting of spots in a 
plurality of images. 

36. The method of claim 33 wherein the cluster is counted as a number of 
discrete signals indicated by a gain factor applied to the number of discrete signals 
in the set. 

37. The method of claim 1 5 wherein the plurality of images are a set of 
images taken during a first analysis of a first color channel, and a second set of 
images are taken of the biological sample for a second color channel, the method 
further comprising: 

avoiding counting discrete signals appearing at a same location in the set of 
images for the first color channel and the set of images in the second color channel. 

38. The method of claim 1 5 wherein the plurality of images are a set of 
images taken for a test probe, and a second set of images are taken of the biological 
sample for a reference probe, the method further comprising: 

avoiding counting discrete signals appearing at a same location in the set of 
images for the test probe and the set of images for the reference probe. 

39. The method of claim 1 5 further comprising: 

receiving a directive from a user indicating counting is to be avoided for a 
specified portion of the biological sample; and 

responsive to the directive, avoiding counting discrete signals for the 
specified portion of the biological sample. 

40. The method of claim 1 5 further comprising: 

receiving a directive from a user indicating counting is to be performed 
separately for a specified portion of the biological sample; and 
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responsive to the directive, separately counting discrete signals for the 
specified portion of the biological sample. 

41 . A high-throughput method of counting fluorescent in situ 
hybridization signals, comprising: 

5 providing an array of biological samples; 

hybridizing the biological samples with a fluorescent test probe that 
hybridizes to a gene of interest in the biological samples and with a fluorescent 
reference probe that hybridizes to a chromosomal reference locus in the biological 
samples; 

10 obtaining images by confocal microscopy of contiguous sections at different 

depths of a plurality of the biological samples; 

detecting fluorescent signal segments from the contiguous sections, and 

segmenting contiguous signals in different contiguous sections into a corresponding 

single spot signal; and 
15 projecting each spot signal into a two-dimensional plane, and counting the 

spots. 

42, The method of claim 41, wherein the fluorescent spot signals from 
the test probe are obtained and detected separately from the fluorescent spot signals 
from the reference probe. 

20 43. The method of claim 42, wherein the fluorescent spot signals from 

the test probe are obtained and detected simultaneously with the fluorescent spot 
signals from the reference probe. 

44. The method of claim 41, wherein the array comprises an array of at 
least 6 different specimens. 
25 45. The method of claim 44, wherein fluorescent in situ hybridization 

signals.are counted in at least 50 different specimens. 

46. The method of claim 41 , wherein the method is a computer 
implemented system. 
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47. The method of claim 41, wherein providing an array of biological 
samples comprises providing an array of biological specimens: 

obtaining a plurality of donor specimens; 

placing each donor specimen in an assigned location in a recipient array; and 
5 obtaining a plurality of copies of the recipient array in a manner that each 

copy contains a plurality of donor specimens that maintain their assigned locations. 

48. A method for counting signals from in situ hybridization of probes in 
biological tissue, the method comprising: 

obtaining digital images of different levels of the biological tissue with a 
10 confocal microscope; 

detecting spot signals at the different levels and separating overlying signals 
from one another. 

49. The method of claim 48, wherein the spot signals are detected at 
different levels with a computer implemented system that detects fluorescent signals 

15 as signal segments from the different levels, wherein the levels are contiguous 

optical sections, and the system segments contiguous signal segments from different 
vertical levels into one spot signal, while separating non-contiguous signal segments 
from different levels into different spot signals. 

50. The method of claim 49, wherein the computer implemented system 
20 projects the spot signals into a single level and counts the spot signals. 

5 1 . The method of claim 49, wherein the computer implemented system 
detects fluorescent signals as spot signals by 

(a) performing a morphological transform to digital images of the different 
levels to obtain digital representations of brightness intensity that indicate a signal 

25 segment; 

(b) eliminating signal segments below a threshold; 

(c) grouping contiguous signal segments into a single spot. 

52. The method of claim 51, further comprising: 

avoiding signal segments having an intensity above a threshold. 
30 53. The method of claim 51, further comprising: 

identifying clusters within the signal segments; and 
counting the clusters each as a plurality of spots. 



54. A device for counting signals from in situ hybridization of probes in 
biological tissue, the device comprising: 

a confocal microscope; 

a digital camera positioned to obtain digital optical sections of different 
5 levels of the biological tissue; and 

a computer implemented system that detects brightness signals at the 
different levels and separates overlapping brightness signals from one another. 

55. The device of claim 54, wherein the computer implemented system 
detects fluorescent signals as signal segments from the different levels, wherein the 

10 levels are contiguous, and the system groups contiguous signal segments from 
different levels into one spot signal, while separating non-contiguous signal 
segments from different levels into different spot signals. 

56. The device of claim 55, wherein the computer implemented system 
counts the spot signals. 

15 57. The device of claim 55, wherein the computer implemented system 

detects fluorescent signals as spot signals by 

(a) performing a morphological transform to digital images of the different 
levels to obtain a digital representation of brightness intensity that indicate a signal 
segment; 

20 (b) eliminating signal segments below a threshold; 

(c) segmenting contiguous signal segments; and 

(d) grouping vertical contiguous signal segments into a single spot signal 
at an assigned level associated with a greatest signal segment intensity. 

58. A computer-readable medium comprising computer-executable 
25 instructions for performing the following; 

within a stack of image slices generated from a plurality of confocal 
microscopic observations of a FISH experiment as a plurality of depths along a z- 
axis, identifying possible fluorescent image components; 

projecting the possible fluorescent image components within the image 
30 slices onto a projection image; 

discarding insignificant contiguous possible fluorescent image 
components in the slices; 
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for each contiguous region in the projection image, grouping regions of 
possible florescent image components associated with the contiguous region in the 
projection image into spot candidates; 

applying a filter to the spot candidates; and 
5 counting the remaining spot candidates as spots. 

59. The computer-readable medium of claim 58 wherein insignificant 
contiguous possible fluorescent image components are determined by comparing a 
size of a contiguous possible fluorescent image component with a threshold. 

60. A computer-generated user interface for presenting results of 

10 microscopic observation of biological tissue subjected to a FISH experiment, the 
user interface comprising: 

a scatter plot of sets of image components designated as spot candidates for 
the FISH experiment; 

wherein the scatter plot comprises points indicating a size and intensity of 
15 spot candidates. 



61 . The computer-generated user interface of claim 60 wherein at least one 
point is operable to receive a user interface activation to navigate to a user interface 
display of information for a spot candidate associated with the point. 
20 62. The computer-generated user interface of claim 60 wherein at least one 

point is operable to receive a user interface activation to navigate to a user interface 
display of a three-dimensional depiction of a spot candidate associated with the 
point. 

63. The computer-generated user interface of claim 60 further comprising: 
25 a display image depicting a view of the tissue subjected to the FISH 

experiment and a depiction of least one candidate spot thereon; 

wherein the depiction of the candidate spot is operable to receive a user 
interface activation to designate the spot as a minimal intensity spot; and 

wherein candidate spots designated as minimal intensity spots are visually 
30 emphasized when presenting the scatter plot. 
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SIGNAL COUNTING FOR IN SITU HYBRIDIZATION 

ABSTRACT 

Fluorescently tagged nucleic acid probe signals are counted in 
5 biological specimens by determining a ratio of signals from a test probe to 

signals of a reference probe. Probe signals need not be counted with 
reference to cells, nuclei, or nuclear contours. Gene amplification or deletion 
can thus be detected by analyzing the ratio. Successive image slices are 
obtained by confocal microscopy, and the images are digitized. The digital 
10 images are transformed and analyzed to combine contiguous fluorescent 

signal segments in successive optical sections to identify discrete probe 
signals, or spots. Spots overlapping in the axial and transverse dimensions of 
a three-dimensional representation of the biological specimens can be 
distinguished. A graphical user interface presents various features for 
15 consideration by a user, who can provide guidance to a computer system 

counting the spots. Various features directed to identifying spot clusters and 
autofluorescent material can increase accuracy of spot counting. 
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Attorney Ret No, 43235-62295 

COMBINED DECLARATION AND POWER OF ATTORNEY 
FOR PATENT APPLICATION 

As a below named inventor, I hereby declare thai: 

My residence, post office address and citizenship are as stated below next to my name. 

I believe I am an original, first and joint inventor (If plural names are listed below) of the subject 
matter which is claimed and for which a patent is sought cm the invention entitled, the specification of which 

El was filed on March 13.2002 as United States Application No. 10/0S8.269 . 

KJ was described and claimed In PCT International Application No, PCTAJS0l'/254fi5, filed on 
September 15, 2000, and amended by the Preliminary Amendment filed on March IS, 200(2, 

I hereby state that I have reviewed and understand the contents of the above- identified specification, 
including the claims, as emended by any amendment referred to above. 

1 acknowledge the duty to disclose information which is material to patentability as defined in Title 
37, Code of Federal Regulations, § 1,56. If this is a continuation-in-part application filed under the conditions 
specified in 35 US.C. § 120 which discloses and claims subject matter in addition to thfU disclosed in the prior 
copending application, I further acknowledge the duty to disclose material information S3 defined in 37 CFR 
§ 1 .56 which occurred between the filing date of the prior application and the national or PCT international 
filing date of the continuation-in-part application. 

1 hereby claim foreign priority benefits under Title 33, United States Code, U9Ca>(d) of any 
foreign application(s) for patent or inventor's certificate or of an PCT International app}xcati0fl(s) designating at 
least one country other than die United States of America listed below and have also identified below any foreign 
appHcation(s) for patent Or inventor^ certificate or any PCT International application (a) designating at least one 
country other than the United States of America filed by me on the same subject matter having a filing date 
before that of the application^) on which priority is claimed: 



Prior Foreign AppUcation(s) 


Country 


Filing Date 


Priority Claimed 








□ Yes □ No 



I hereby claim the benefit under Title 35. United States Code, $ 1 19(e) of any United States 
provisional applicationf s) listed below: 



Application No. 


Filing Date 


60/154.601 


September 17, 1999 



I hereby claim the benefit under Title 35, United States Code, § 120 of any United States 
application^) or § 365(c) of any PCT International application^) designating the United States, listed below 
and, insofar as the subject matter of each of the claims of this application Is not disclosed in the prior United 
States or PCT International application in the manner provided by the first paragraph of Title 35, United States 
Code* § 112, 1 acknowledge the duty to disclose material information as defined in Title 37, Code of Federal 
Regulations, § l.56Xa) which occurred between the filing date of the prior application amd the national or PCT 
International filing date of this application: 



Application No. 


Filing Date 


Status; patented, pending, abandoned 
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corresponding international application, and to transact all business in the Patent and Trademark Office 
connected therewith: 
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Jack Spiegel 
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Devon J. Zastrow 



Xieg< No. 
34,477_ 
32.808 
38,448 

46,005 



Reg* No* 

42,630 



47 An 

50,206 



all of the law firm of Kla/miist Sparkman, LDP. 

Address all telephone calls to Gregory L. Maurer, telephone number 503/2215-7391 and raa>ImUe 
number 303/228-9446. 

Address all correspondence to; 

One World Trad e Center, Suite 1 600 
121 SW S almon Street—^ 
* Portland. OR 97204-59^8 



I hereby declare that ail statements made herein of my own knowledge are true end mat all 
made on Information and belief are believed to be true; and further that these statements were made - ' 
knowledge that willful raise statements and the like so made are punishable by fine or imprisonment, 
under Section 1001 of Title 18 of the United States Code end that such willful retse statements may j 
the validity of the application or any potent leaned thereon. 
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